[MUSIC]

Working on the web.

It seems like there's always an API that you need to integrate with for

your next project.

Maybe you want to use the New York Public Library's API, or

Marvel's superhero database.

Or maybe you just need to tie into something more mundane,

like your local bus service or Akamai.

Regardless of the web based API, the super powerful and

popular request project has you covered.

You'll need to install it with pip install requests.

Let's see how to use this excellent tool.

I think that using the request library is a pleasure.

Mostly it's because it has a pretty straightforward API.

So let's play with it a little bit.

I want to make a get request against a server, and

the server I'm going to be using is this httpbin.org,

which is an awesome, awesome service for testing out HTTP stuff.

So yeah, I'm going to use this for testing out stuff.

We're going to use this for all of our requests except for one.

So, what do we have to do?

Well I'm in the python shell, and what I need to do is, I need to import requests.

So I have it, of course we have to import to the library.

So to make a get request, we actually have to use the get method.

So we're going to say r=requests Dot get, and we're going to get

httpbin dot org slash get, which is this end point.

You see right here, it's an end point for testing get data.

Okay? Cool.

So we got it.

So what do we have?

Well, what we have Is a response object, and you can see that our

response object has a status code of 200, its in the brackets there, 200.

We can check that as an attribute as well, and we get back 200.

And we can actually do r.ok, and it comes back as true.

The cool thing here is that you don't have to check against all of the 200 status

codes, because those are all, like, good status codes.

You can just do r.ok and it says, yeah everything’s cool, everything’s fine.

All right, and then let’s look at our headers.

Let’s see what headers came out.

So there’s our headers, and it’s basically a dictionary, so

we can see all of our different headers.

Like let’s see if we can do headers.keys.

Okay, r.headers Content-Length.

We're working on the web, right?

And on the web, you encounter APIs.

And most of the APIs are going to take and give back JSON data.

So how do we deal with getting back JSON data from an API through

the requests library?

Sadly we can't get interesting json from httpbin without doing some extra work

ourselves and, you know, we don't want to do that extra work if we can help it.

So we're going to use the public GitHub API for viewing of it.

Let's hit that one up real quick.

So we're going to do requests.get

https://api.github.com/events.

Okay, cool.

So now, I have another response object that's the same as my

last response object.

It's got a status code and all that stuff.

But his one has a payload of trace on data.

So let's look at that in it's raw form, and we'll do r.txt,

and oh my god, so much text.

Right?

That's a giant, giant string.

Now, we could import the json library and convert this from a string to

python list of dictionaries, but that seems like a lot of work, doesn't it?

We just went over being lazy.

So, the PO behind the requests library want us to be as lazy as we want to be.

They don't want us to do much of extra work so

they gave us a really handy method that's called JSON.

We can do r.json, and we get all this as JSON data.

Let's make a new variable here, timeline equals r.json.

And then let's do timeline zero, and then let's get the ID key.

And there we go we've got this ID 275 so on.

If you're working with a JSON API a lot this dot JSON method is a huge time saver.

There is also a content.

If we look at that, it's the same thing, basically.

In this case, we have text.

If you're working with binary content, like images, audio, pdfs, what

you'll think of as being a graphical file, or anything that isn't just plain text.

The content comes across as bytes, not text so you

can take those bytes and turn them into your PDF or your SVG image, or whatever.

That's beyond the scope of this workshop, we'll talk about working with images

in Python in another workshop, but it's good to know that it's there.

At some point, you're going to want to post data to a server.

For example, you'll eventually need to create a new object through an API and

usually that requires a post request, and some sort of payload of data.

Like, we got the json payload earlier.

Now you need to send out a payload.

Requests makes this pretty simple, thankfully.

Let's make a payload.

Usually these are dictionaries.

We're going to say content and I really like requests.

And then we'll say user ID is number one fifty two.

All right, nothing special there, just a plain, everyday python dictionary.

So now, let's do R equals requests dot post and we want to send it somewhere.

We're going to send it to httpbin.org/post.

And we're going to say params=payload.

And when we get that back, then we can see that everything's still good.

All right.

We got all the good stuff that we want.

And what's cool is we posted it we don't see in our headers the stuff that came in.

But if we look at our and we do our .text I think.

Yeah. So, we can see here we have these args,

an insight args we have, actually let's do json.

There we go.

So we can see here,

we have args, and we got content requests, and the user ID 52.

So that was what went across, they send that data back to us.

So that's pretty nice.

Sometimes though, you need to submit data inside your payload,

where your key represents a list.

So, this is a little bit tricky, so, I wanted to point this one out specifically.

So, lets make a new payload.

Payload. Actually, you know what,

I'm going to go ahead and exit out of this.

And clear.

Get back in.

Okay, so let's do a new payload here and we're going to say posts.

We're going to mark it as being a list.

I mean sort of, kind of.

Right?

Then, we're going to give it numbers one, two, three and four, five six.

So, those are our posts that we care about.

You know what let's do this as a delete request.

So now we would do r equals requests.delete.

http://httpbin.org/delete.

And params equals payload.

And now if we do r.json and

we look for args and we've got posts one two three,

four five six, what if I do r.headers?

Yeah, that doesn't go back in headers.

So, that's great.

We're able to send our information through and

then we know that it works because it comes back to us.

Good URLs don't change, but every once in a while things can't be good URLs.

So what if your endpoint, your resource that you're posting to, getting from,

whatever, has moved?

Well, requests actually handles this pretty well out of the box.

So, let's just do a pretty basic one,

request.get http://httpbin.org

And we're going to say redirect, and we want it to redirect us three times.

All right, so now we look at the status code, and it's a 200.

So our redirects worked, or it didn't happen.

But we're going to assume that our redirects worked [LAUGH].

So now let's look a r.history.

Look at that.

We've got three response objects because we went to three different things.

So that's great. We got redirected three times,

we told it to that's what happened and

in the end we got a 200 because that's what we wanted.

What if we didn't want to allow any redirects?

What if we wanted to stop it before the redirects happened?

Well we can do that really simply and you may be at, why would we want to do that?

Well maybe you've got all your URLs stored in a database, right?

A URL comes up as being a redirect or as being bad, you want to change that URL.

So let's go back up here and we will do this one again, but

this time we're going to say allow_redirects equals false.

So no redirects.

And if we look at r.statuscode we get a 302.

And if we look at the headers, then we can see that it's sending us to

relative redirect two, which makes sense, we're on three, we'd go to two, go to one.

Then we'd be there.

And we can see that, yeah, so the 302 doesn't come through here, but

we have the status code.

So, that's cool.

We can see where it's trying to send us to.

We can then take that location, update our database,

follow that again maybe, make sure it doesn't redirect as well, and keep going.

But that's all business logic, that's not part of this.

And the last thing I wanted to talk about with requests is doing authentication.

What if you need to authenticate before you can get to the resource?

Now, I would say this is more likely to be done when you're

trying to request a website, a webpage, and you're not trying to hit an API.

APIs generally do authentication through public and private keys.

But a webpage you'll often find it's behind,

say, http basic or http digest authentication.

Both of those authentication methods work about the same, but

http basic authentication sends your password in plain text and

http digest authentication hashes it before it sends it out.

Digest is a little safer, obviously.

But, it seems like you're encounter basic more often than you encounter Digest.

People just don't always think about security.

But, the cool thing is, we can do both of these right inside requests.

So, let's see how to do our basic one first.

From requests.

.Auth, import

HTTPBasicAuth, and

you know what, let's go ahead and import HTTPDigestAuth too,

because I know we're going to use that in a minute.

Okay.

So r = request.get("http://httpbin.org /basic-auth/user/password.

So here, if you look over at httpbin, look at basic auth,

we supply the user and the password that we want them to come up with.

And then we're going to say our auth equals HTTPBasicAuth.

And then we have to supply our user name and our password.

All right.

So let that run through and then we check out the status code.

Our status code is 200.

So our auth worked.

If we were to try that again and

we'll say like password2 and now we look at status code.

We see that it failed.

We got a 401, which is a forbidden.

So, cool, that's basic auth.

You supply your user name and your password as auth equals HTTPBasicAuth and

you go to whatever the URL is that requires basic auth.

You may be surprised to know that the digest auth is pretty similar.

So, let's try that one.

request.get('http://httpbin.org/digest-au- th/auth/user/password') and

our auth is HTTPDigestAuth, again, with user and password.

Okay. Only thing we changed here is the digest

stuff and our url changed a little bit.

All right? And r.status_code is again 200.

And again if we wanted to try this with a bad password, then we'd get a 401.

Request has support for authentication through OAuth and

people submitted extra packages for

authentication through Kerberos and NTLM and also some other ones.

I'll have a link in the teachers notes to request

docs which has much more information about all of this.

And even stuff we didn't cover the request library

is quickly going to become an indispensable part of your work flow.

If you do a lot of work with APIs or HTTP requests.

If you're going to be using for a large work flow, be sure and

check out these session object that request provides.

It will make working with cookies and

a lot of other persistent parameters a lot lot easier.

If you liked this format, be sure to let us know through email and the forum.

Also, let us know what other topics you'd like covered in future workshops.

Thanks for watching, and I'll see you next time.