Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
The requests library is one of the most popular Python libraries, period. It's amazingly friendly and useful and makes working with RESTful APIs on the Internet a breeze. In this workshop, we'll look at how to use the library to make GET, POST, and other requests, how to process JSON data, and how to handle HTTP Basic and Digest auths.
pip install requests
Documentation for Requests.
requests.get('<url>') # GET request
requests.post('<url>', params={}) # POST request with payload
Related Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign upRelated Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign up
[MUSIC]
0:00
Working on the web.
0:04
It seems like there's always an API
that you need to integrate with for
0:04
your next project.
0:08
Maybe you want to use
the New York Public Library's API, or
0:09
Marvel's superhero database.
0:12
Or maybe you just need to tie
into something more mundane,
0:14
like your local bus service or Akamai.
0:17
Regardless of the web based API,
the super powerful and
0:19
popular request project has you covered.
0:22
You'll need to install it
with pip install requests.
0:25
Let's see how to use this excellent tool.
0:28
I think that using the request
library is a pleasure.
0:30
Mostly it's because it has
a pretty straightforward API.
0:33
So let's play with it a little bit.
0:36
I want to make a get request
against a server, and
0:37
the server I'm going to be
using is this httpbin.org,
0:41
which is an awesome, awesome service for
testing out HTTP stuff.
0:46
So yeah, I'm going to use this for
testing out stuff.
0:52
We're going to use this for
all of our requests except for one.
0:53
So, what do we have to do?
0:57
Well I'm in the python shell, and what I
need to do is, I need to import requests.
0:59
So I have it, of course we
have to import to the library.
1:06
So to make a get request,
we actually have to use the get method.
1:10
So we're going to say r=requests Dot get,
and we're going to get
1:14
httpbin dot org slash get,
which is this end point.
1:20
You see right here,
it's an end point for testing get data.
1:26
Okay?
Cool.
1:30
So we got it.
1:31
So what do we have?
1:31
Well, what we have Is a response object,
and you can see that our
1:32
response object has a status code of 200,
its in the brackets there, 200.
1:37
We can check that as an attribute as well,
and we get back 200.
1:41
And we can actually do r.ok,
and it comes back as true.
1:47
The cool thing here is that you don't have
to check against all of the 200 status
1:51
codes, because those are all,
like, good status codes.
1:54
You can just do r.ok and it says,
yeah everything’s cool, everything’s fine.
1:58
All right, and
then let’s look at our headers.
2:01
Let’s see what headers came out.
2:05
So there’s our headers, and
it’s basically a dictionary, so
2:07
we can see all of our different headers.
2:10
Like let’s see if we can do headers.keys.
2:12
Okay, r.headers Content-Length.
2:16
We're working on the web, right?
2:23
And on the web, you encounter APIs.
2:24
And most of the APIs are going to take and
give back JSON data.
2:27
So how do we deal with getting
back JSON data from an API through
2:32
the requests library?
2:36
Sadly we can't get interesting json
from httpbin without doing some extra work
2:38
ourselves and, you know, we don't want
to do that extra work if we can help it.
2:43
So we're going to use the public
GitHub API for viewing of it.
2:47
Let's hit that one up real quick.
2:52
So we're going to do requests.get
2:55
https://api.github.com/events.
2:59
Okay, cool.
3:04
So now, I have another response
object that's the same as my
3:05
last response object.
3:09
It's got a status code and all that stuff.
3:11
But his one has a payload
of trace on data.
3:12
So let's look at that in it's raw form,
and we'll do r.txt,
3:16
and oh my god, so much text.
3:20
Right?
3:22
That's a giant, giant string.
3:23
Now, we could import the json library and
convert this from a string to
3:25
python list of dictionaries, but
that seems like a lot of work, doesn't it?
3:30
We just went over being lazy.
3:35
So, the PO behind the requests library
want us to be as lazy as we want to be.
3:37
They don't want us to do
much of extra work so
3:42
they gave us a really handy
method that's called JSON.
3:45
We can do r.json, and
we get all this as JSON data.
3:48
Let's make a new variable here,
timeline equals r.json.
3:53
And then let's do timeline zero,
and then let's get the ID key.
3:58
And there we go we've got this ID 275 so
on.
4:05
If you're working with a JSON API a lot
this dot JSON method is a huge time saver.
4:08
There is also a content.
4:16
If we look at that,
it's the same thing, basically.
4:20
In this case, we have text.
4:23
If you're working with binary content,
like images, audio, pdfs, what
4:24
you'll think of as being a graphical file,
or anything that isn't just plain text.
4:28
The content comes across as bytes,
not text so you
4:33
can take those bytes and turn them into
your PDF or your SVG image, or whatever.
4:38
That's beyond the scope of this workshop,
we'll talk about working with images
4:43
in Python in another workshop, but
it's good to know that it's there.
4:46
At some point, you're going to
want to post data to a server.
4:50
For example, you'll eventually need to
create a new object through an API and
4:54
usually that requires a post request,
and some sort of payload of data.
4:58
Like, we got the json payload earlier.
5:02
Now you need to send out a payload.
5:05
Requests makes this pretty simple,
thankfully.
5:07
Let's make a payload.
5:10
Usually these are dictionaries.
5:11
We're going to say content and
I really like requests.
5:13
And then we'll say user ID
is number one fifty two.
5:19
All right, nothing special there,
just a plain, everyday python dictionary.
5:23
So now, let's do R equals requests dot
post and we want to send it somewhere.
5:29
We're going to send it
to httpbin.org/post.
5:36
And we're going to say params=payload.
5:39
And when we get that back, then we
can see that everything's still good.
5:46
All right.
5:55
We got all the good stuff that we want.
5:55
And what's cool is we posted it we don't
see in our headers the stuff that came in.
5:58
But if we look at our and
we do our .text I think.
6:05
Yeah.
So, we can see here we have these args,
6:08
an insight args we have,
actually let's do json.
6:13
There we go.
6:15
So we can see here,
6:16
we have args, and we got content requests,
and the user ID 52.
6:19
So that was what went across,
they send that data back to us.
6:24
So that's pretty nice.
6:28
Sometimes though, you need to
submit data inside your payload,
6:32
where your key represents a list.
6:37
So, this is a little bit tricky, so, I
wanted to point this one out specifically.
6:41
So, lets make a new payload.
6:45
Payload.
Actually, you know what,
6:47
I'm going to go ahead and
exit out of this.
6:48
And clear.
6:52
Get back in.
6:52
Okay, so let's do a new payload here and
we're going to say posts.
6:57
We're going to mark it as being a list.
7:02
I mean sort of, kind of.
7:04
Right?
7:05
Then, we're going to give it numbers one,
two, three and four, five six.
7:06
So, those are our posts
that we care about.
7:11
You know what let's do
this as a delete request.
7:14
So now we would do r
equals requests.delete.
7:18
http://httpbin.org/delete.
7:21
And params equals payload.
7:26
And now if we do r.json and
7:28
we look for args and
we've got posts one two three,
7:35
four five six, what if I do r.headers?
7:40
Yeah, that doesn't go back in headers.
7:44
So, that's great.
7:46
We're able to send our
information through and
7:47
then we know that it works
because it comes back to us.
7:49
Good URLs don't change, but every once
in a while things can't be good URLs.
7:53
So what if your endpoint, your resource
that you're posting to, getting from,
7:59
whatever, has moved?
8:03
Well, requests actually handles
this pretty well out of the box.
8:06
So, let's just do a pretty basic one,
8:09
request.get http://httpbin.org
8:14
And we're going to say redirect, and
we want it to redirect us three times.
8:20
All right, so now we look at
the status code, and it's a 200.
8:25
So our redirects worked,
or it didn't happen.
8:29
But we're going to assume that
our redirects worked [LAUGH].
8:33
So now let's look a r.history.
8:36
Look at that.
8:38
We've got three response objects because
we went to three different things.
8:40
So that's great.
We got redirected three times,
8:45
we told it to that's what happened and
8:47
in the end we got a 200
because that's what we wanted.
8:49
What if we didn't want
to allow any redirects?
8:52
What if we wanted to stop it
before the redirects happened?
8:54
Well we can do that really simply and you
may be at, why would we want to do that?
8:58
Well maybe you've got all your
URLs stored in a database, right?
9:02
A URL comes up as being a redirect or
as being bad, you want to change that URL.
9:05
So let's go back up here and
we will do this one again, but
9:11
this time we're going to say
allow_redirects equals false.
9:15
So no redirects.
9:21
And if we look at
r.statuscode we get a 302.
9:24
And if we look at the headers,
then we can see that it's sending us to
9:27
relative redirect two, which makes sense,
we're on three, we'd go to two, go to one.
9:33
Then we'd be there.
9:36
And we can see that, yeah, so
the 302 doesn't come through here, but
9:38
we have the status code.
9:43
So, that's cool.
9:44
We can see where it's
trying to send us to.
9:45
We can then take that location,
update our database,
9:47
follow that again maybe, make sure it
doesn't redirect as well, and keep going.
9:51
But that's all business logic,
that's not part of this.
9:54
And the last thing I wanted to talk about
with requests is doing authentication.
9:58
What if you need to authenticate
before you can get to the resource?
10:02
Now, I would say this is more
likely to be done when you're
10:06
trying to request a website, a webpage,
and you're not trying to hit an API.
10:10
APIs generally do authentication
through public and private keys.
10:15
But a webpage you'll
often find it's behind,
10:19
say, http basic or
http digest authentication.
10:21
Both of those authentication
methods work about the same, but
10:25
http basic authentication sends
your password in plain text and
10:30
http digest authentication hashes
it before it sends it out.
10:34
Digest is a little safer, obviously.
10:38
But, it seems like you're encounter basic
more often than you encounter Digest.
10:40
People just don't always
think about security.
10:45
But, the cool thing is, we can do
both of these right inside requests.
10:48
So, let's see how to do
our basic one first.
10:52
From requests.
10:56
.Auth, import
10:58
HTTPBasicAuth, and
11:01
you know what, let's go ahead and
import HTTPDigestAuth too,
11:07
because I know we're going
to use that in a minute.
11:13
Okay.
11:18
So r = request.get("http://httpbin.org
/basic-auth/user/password.
11:18
So here, if you look over at httpbin,
look at basic auth,
11:26
we supply the user and the password
that we want them to come up with.
11:32
And then we're going to say our
auth equals HTTPBasicAuth.
11:37
And then we have to supply our
user name and our password.
11:46
All right.
11:52
So let that run through and
then we check out the status code.
11:53
Our status code is 200.
11:55
So our auth worked.
11:57
If we were to try that again and
11:59
we'll say like password2 and
now we look at status code.
12:02
We see that it failed.
12:07
We got a 401, which is a forbidden.
12:08
So, cool, that's basic auth.
12:10
You supply your user name and your
password as auth equals HTTPBasicAuth and
12:12
you go to whatever the URL
is that requires basic auth.
12:17
You may be surprised to know that
the digest auth is pretty similar.
12:20
So, let's try that one.
12:24
request.get('http://httpbin.org/digest-au-
th/auth/user/password') and
12:26
our auth is HTTPDigestAuth,
again, with user and password.
12:36
Okay.
Only thing we changed here is the digest
12:45
stuff and our url changed a little bit.
12:47
All right?
And r.status_code is again 200.
12:51
And again if we wanted to try this with
a bad password, then we'd get a 401.
12:53
Request has support for
authentication through OAuth and
13:01
people submitted extra packages for
13:04
authentication through Kerberos and
NTLM and also some other ones.
13:07
I'll have a link in
the teachers notes to request
13:12
docs which has much more
information about all of this.
13:15
And even stuff we didn't
cover the request library
13:18
is quickly going to become
an indispensable part of your work flow.
13:21
If you do a lot of work with APIs or
HTTP requests.
13:24
If you're going to be using for
a large work flow, be sure and
13:27
check out these session
object that request provides.
13:29
It will make working with cookies and
13:32
a lot of other persistent
parameters a lot lot easier.
13:34
If you liked this format, be sure to let
us know through email and the forum.
13:38
Also, let us know what other topics
you'd like covered in future workshops.
13:41
Thanks for watching, and
I'll see you next time.
13:45
You need to sign up for Treehouse in order to download course files.
Sign upYou need to sign up for Treehouse in order to set up Workspace
Sign up