A Beginners Guide to requests Library in Python : How to Use requests library in Python

Most existing Python modules for sending HTTP requests are extremely verbose and cumbersome. Python’s builtin urllib2 module provides most of the HTTP capabilities you should need, but the api is thoroughly broken. It requires an enormous amount of work (even method overrides) to perform the simplest of tasks.
Requests is an Apache2 Licensed HTTP library, written in Python. It is designed to be used by humans to interact with the language. This means we don’t need have to manually add query strings to URLs, or form-encode your POST data. Requests will allow to send HTTP/1.1 requests using Python. With it, we can add content like headers, form data, multi-part files, and parameters via simple Python libraries. It also allows to access the response data of Python in the same way.

Features :
  • Keep-Alive & Connection Pooling
  • Sessions with Cookie Persistence
  • Browser-style SSL Verification
  • Basic/Digest Authentication
  • Elegant Key/Value Cookies
  • Automatic Decompression
  • Unicode Response Bodies
  • Multipart File Uploads
  • Connection Timeouts
  • Thread-safety
  • HTTP(S) proxy support

Installation :

To install requests library use the below command
 sudo apt-get install python-requests
or install with pip
 pip install requests
 easy_install requests

HTTP GET requests :

Start python interactive shell, import requests library by 'import requests' and let start
 re = requests.get('http://sec-art.net')
Now all the information about our request is now stored in a Response object re. Now use the below commands for request details
 re.status_code # to check the status code
We can also check the status code with requests built-in lookup codes
 re.status_code == requests.codes.ok

 re.encoding  # to check encoding property
 re.url   # return rul
 re.text  # shows the response body if it is text (parse it as unicode) 
 re.content  # show the reponse body
 re.headers  # show all the headers as key->value pairs
 re.cookies  # show all cookies
we can also use other HTTP request types, like PUT, DELETE, HEAD and OPTIONS
 r = requests.put("http://httpbin.org/put")
 r = requests.delete("http://httpbin.org/delete")
 r = requests.head("http://httpbin.org/get")
 r = requests.options("http://httpbin.org/get")
Now to see individual headers
 re.headers['Content-Length']  # show the content length
 re.headers['Content-Type'] # show the content type
for other headers just put the name of the header name.

Sending requests with Parameters :

First we need to create a dictonary of parameters
 p = {'name' : 'Frank Castle', 'Age' : '40', 'Occupation' : 'Self-Employed'}
 re = requests.get('http://google.com', params=p)
We can check the url by
Another example
 P = { 'search_query' : 'latest movies'}
 re = requests.get('https://youtube.com', params=p)

HTTP POST Requests :
 import requests
 re = requests.post('https://dvwa.com/Login.php', data={'username':'Frank', 'password':'12345', 'Submit':'Login'})
The post data can be sent as dictonary named data. To send post param in json format
 import json
 import requests
 payload = {'username':'Frank', 'password':'12345', 'Submit':'Login'}
 re = requests.post('https://dvwa.com/Login.php', data=json.dumps(payload))

Sending Custom Headers :
 import requests
 headers = {'Custom1': 'test101', 'Custom2': 'exploit', 'Custom3': 'hello_world'}
 re = requests.post('https://requestb.in/135oaie1', headers=headers)

Sending Custom Cookies  :
 import requests
 cookies = {'Custom1': 'test101', 'Custom2': 'exploit', 'Custom3': 'hello_world'}
 re = requests.post('https://requestb.in/135oaie1', cookes=cookies)

Session Objects

To maintain the same session within different requests
 import requests
 session1 = requests.Session()
 re1 = session1.get('http://facebook.com/somepage')      # first request 
 re2 = session1.get('http://facebook.com/someotherpage') # second request 


To set timeout in requests
 import requests
 re =  requests.get('http://sec-art.net/', timeout=3)
 re =  requests.get('http://sec-art.net/', timeout=0.001)

For more information visit below links :