Bring Your Own User-Agent

class:  center, middle
name:   title

# Bring Your Own User-Agent

Charles McGarvey

???
Hi, I'm Chaz.

I want to talk to you about HTTP user agents.

---
class: img-map, center, middle

![WebService modules on CPAN](img/webservice-on-cpan.png)

???

There is a whole class of so-called "web service" modules on CPAN that provide a nice perly interface for interacting
with... web services.

All kinds of things, from...

---
class: img-map, center, middle

![Twilio module](img/twilio.png)

???
to

---
class: img-map, center, middle

![Ontario Power Generation module](img/opg.png)

???
the Ontario Power Generation website.

Random, but why not?

---
class: img-map, center, middle

![WebService modules on CPAN](img/webservice-on-cpan-circled.png)

???
Most of these modules congregate here (or at least they should).

---
class: center, middle

## `WebService`

???
In the `WebService` namespace.

--
## `Net`

???
There are other common namespaces for this sort of thing...

--
## `WWW`

---
class: center, middle

## `WebService`
## <strike>`Net`</strike>
## <strike>`WWW`</strike>

???
PSA: For new stuff, prefer the `WebService` namespace for such modules.

---
class: center, middle

## `WebService` modules are useful.

???
Even though a lot of APIs nowadays are RESTful which may be easy to use with just your favorite user agent, these
modules often take care of some of the tricky or boring details, like:

- authentication
- paging

Details that are important but you don't want to read through the API documentation to figure it out.

---
name: but
class: center, middle

But

???
And here's the problem...

---
class: center, middle

## These modules are .major-em[**tightly-coupled**] to specific .major-em[user agents].

### ;-(

---
class: center, middle

![Dependencies](img/deps1.png)
![Dependencies](img/deps2.png)
![Dependencies](img/deps3.png)
![Dependencies](img/deps4.png)
![Dependencies](img/deps5.png)

???
Most of them use `LWP` or `HTTP::Tiny`.

---
class: center, middle

## This... could be better.

???
Now I'm going to try to convince you that this is a problem.

---
class: center, middle

## Problem:

### How to configure the user agent...

???
User agents typically have defaults and so may not need to be configured, but what if the user needs the user agent to
support
- proxying,
- caching,
- TLS verification,
- timeouts...

---
class: ex-code

```perl
use WebService::Whatever;

my $ws = WebService::Whatever->new(verify_SSL => 1);

$ws->timeout(10);

my $resp = $ws->account_info;

...
```

???
So, one way this has been solved is for the webservice to expose all the attributes and knobs needed to configure the
internal agent and just pass them through.

But this kinda bad because now every webservice module has to do this,

...and every module will probably do it slightly differently.

---
class: ex-code

```perl
use HTTP::Tiny;
use WebService::Whatever;

my $ua = HTTP::Tiny->new(verify_SSL => 1);

*my $ws = WebService::Whatever->new(ua => $ua);

$ua->timeout(10);

my $resp = $ws->account_info;

...
```

???
Then someone remembered that dependency injection was a thing, so now we have modules that let you pass in your own user
agent.

This is great because it lets the user set up the user agent however they want, and webservice modules writers don't
need to do any of that boring work.

Big improvement!

---
class: ex-code

```perl
*use Mojo::UserAgent;
use WebService::Whatever;

*my $ua = Mojo::UserAgent->new(insecure => 0);

my $ws = WebService::Whatever->new(ua => $ua);

$ua->connect_timeout(10);

my $resp = $ws->account_info;       # ;-(

...
```

???
But I can't just plug in any user agent I want!

If the webservice module was written for `HTTP::Tiny` or any other user agent, it's expecting that I'm going to pass it
the kind of user agent it wants.

This makes me sad.

---
class: center, middle

## Let the user decide.

???
I think the user writing a program should decide which user agent to use.

After all, they're the ones who know the requirements of their program.

If I'm writing a program that needs to use the least amount of resources, and I want to use a webservice that is coupled
with a *not* tiny user agent, then I'm out of luck.

Or if somebody wrote a great webservice module using a blocking interface like `HTTP::Tiny` or `LWP` that I want to use
but my program is event-driven and so can't block, then I'm out of luck again.

---
## [`Mail::SendGrid`](https://metacpan.org/pod/Mail::SendGrid) -> [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
## [`Mojo::Sendgrid`](https://metacpan.org/pod/Mojo::Sendgrid) -> [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
## [`WebService::SendGrid`](https://metacpan.org/pod/WebService::SendGrid) -> [`Net::Curl::Simple`](https://metacpan.org/pod/Net::Curl::Simple)
## ...

???
The solution we've come up with so far is to just implement a new webservice module for each type of user agent that
anyone cares to use.

Wasted effort. :-(

---
class: center, middle

## There's a better way.

???
What we need is user agent **adapter**.

As in, the **adapter pattern**, which is the same pattern we generally use for the myriad "Any" modules.

We need something that has an common inteface that module writers can code against, and then adapters to transform the
request and response appropriately for whatever real user agent is wanted.

So yeah, this isn't an original idea of mine.

---
## [`HTTP::Any`](https://metacpan.org/pod/HTTP::Any)

???
I searched CPAN and found just such a thing!

--
#### But it has some .major-em[fatal flaws]...

???
in my opinion. (No offense to this module's author.)

--
### 1. It provides its own *new* interface.

???
- Okay, this one's not fatal.
- Nobody wants to learn yet another user agent interface.
- And it's a callback interface in order to support non-blocking user agents.

But having to set callback functions if you're not actually using a non-blocking user agent is kinda clunky.

--
### 2. It doesn't support many user agents.

???
- `LWP`
- `Curl`
- `AnyEvent`

--
### 3. It doesn't actually provide a common interface.

???
so it's not really usable as an adapter.

There were a couple other potential solutions on CPAN I found, but none of them overcome all of these problems.

Some of the modules that look like they might work at face value, actually are aiming to solve the opposite problem;
that is, when the user doesn't care what user agent is used, just find one and use it.

---
class: center, middle

## I wrote a module.

---
class: center, middle

## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)

???
This one is different because it has a "UA" at the end (for "user agent").

It also solves the problems I had with the other module.

---
## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)

### 1. Uses the `HTTP::Tiny` interface.

???
- So not much new to learn.
- And it doesn't make you use a callback interface if your user agent is non-blocking.

If your webservice module already uses `HTTP::Tiny`, this is *almost* a drop-in replacement.

--
### 2. Supports at least six user agents.

.col[
- [`AnyEvent::HTTP`](https://metacpan.org/pod/AnyEvent::HTTP)
- [`Furl`](https://metacpan.org/pod/Furl)
- [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
]
.col[
- [`LWP::UserAgent`](https://metacpan.org/pod/LWP::UserAgent)
- [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
- [`Net::Curl::Easy`](https://metacpan.org/pod/Net::Curl::Easy)
]

???
Plus any user agent that inherits from any of these in a well-behaved manner should also work.

It's pretty easy to support for other user agents.

--
### 3. Provides a *common* interface.

???
The interface, like I said, is the `HTTP::Tiny` interface.

---
class: ex-code

```perl
has ua => (                 # <-- user agent
    is       => 'ro',
    required => 1,
);

has any_ua => (
    is       => 'lazy',
    default  => sub {
        my $self = shift;
        require HTTP::AnyUA;
        HTTP::AnyUA->new(ua => $self->ua);
    },
);
```

???
A webservice module implementing this looks something like this.
- Allow the user to pass in a useragent. You could also default to `HTTP::Tiny` or something if you wanted the attribute
  to be optional.
- Then you construct an `HTTP::AnyUA` instance and pass it the useragent.

---
class: ex-code

```perl
sub account_info {
    my $self = shift;

*   my $resp = $self->any_ua->get(
        $self->base_url . '/account',
        {
            headers => {
                authorization => $self->auth,
            },
        },
    );

return $resp;
}
```

???
The webservice methods then use the `HTTP::AnyUA` to make requests using the same args and response that `HTTP::Tiny`
has.

---
class: ex-code

```perl
my $ua = HTTP::Tiny->new;

my $ws = WebService::Whatever->new(ua => $ua);

my $resp = $ws->account_info;
```

???
A **user** of the webservice module would look like this.

You just provide the useragent to the webservice.

---
class: ex-code

```perl
my $ua = LWP::UserAgent->new;

my $ws = WebService::Whatever->new(ua => $ua);

my $resp = $ws->account_info;
```

---
class: ex-code

```perl
my $ua = Mojo::UserAgent->new;

my $ws = WebService::Whatever->new(ua => $ua);

my $resp = $ws->account_info;
```

---
class: ex-code

```perl
my $ua = 'AnyEvent::HTTP';

my $ws = WebService::Whatever->new(ua => $ua);

my $resp = $ws->account_info;
```

---
class: ex-code

```perl
my $ua = 'AnyEvent::HTTP';

my $ws = WebService::Whatever->new(ua => $ua);

my $resp = $ws->account_info;

# {
#     success => 1,
#     url     => "https://whatever/account"
#     status  => 200,
#     reason  => "OK",
#     content => "{...}",
#     headers => {...},
# }
```

???
The response from `HTTP::AnyUA` always looks like an `HTTP::Tiny` response, regardless of which user agent the user
provides.

In this case, my "whatever" webservice is just passing the raw response back to the user, but a more useful service will
decode the response content.

And, in the case that the user provides a non-blocking user agent, then instead of returning a hashref with the normal
`HTTP::Tiny` response, it returns a `Future` object that resolves to a hashref with the normal `HTTP::Tiny` response. So
you know what to expect.

---
class: center, middle

![HTTP::AnyUA diagram](img/http-anyua-diagram.svg)

???
I think this is pretty cool already, but I'll show you one more thing before I get kicked off that's even cooler...

When you have a request response pipeline that is shaped like this, it begs to have...

---
class: center, middle

![HTTP::AnyUA with middleware diagram](img/http-anyua-middleware-diagram.svg)

???
Just like in PSGI where you can have middleware components between whatever server handler and the app, you can do the
same sort of thing for the client.

You can write components that work for any user agent and plug them in.

I've written only a couple components,
- one to time or profile each request and
- another to ensure a proper 'content-length' header is set.

Middleware components can do anything, even short-circuit and not actually call the user agent.

I started writing a caching component, but it's taking me awhile to write because I do want it to be
`RFC-7234`-compliant (and my interest jumps around), but it will be cool because not every user agent has a decent
cache.

With HTTP::AnyUA, I just need to implement the cache once and it works for all user agents that can be plugged into this
pipeline.

---
class: center, middle

## Conclusion

???
If you're writing a module that needs to *use* an HTTP user agent but otherwise has no reason to care what the user
agent actually is, consider using `HTTP::AnyUA` or something like it (instead of coupling directly with a user agent).

It will make your webservice module usable by more people.

---
class: center, middle
name:  last

### Thanks.