How to handle proxies in urllib3

python proxy urllib3
Tom picture Tom · Jul 1, 2015 · Viewed 20k times · Source

I am having trouble finding solid examples of how to build a simple script in urllib3 which opens a url (via a proxy), then reads it and finally prints it. The proxy requires a user/pass to authenticate however it's not clear to me how you do this? Any help would be appreciated.

Answer

shazow picture shazow · Jul 1, 2015

urllib3 has a ProxyManager component which you can use. You'll need to build headers for the Basic Auth component, you can either do that manually or use the make_headers helper in urllib3.

All together, it would look something like this:

from urllib3 import ProxyManager, make_headers

default_headers = make_headers(proxy_basic_auth='myusername:mypassword')
http = ProxyManager("https://myproxy.com:8080/", proxy_headers=default_headers)

# Now you can use `http` as you would a normal PoolManager
r = http.request('GET', 'https://stackoverflow.com/')

Related questions

Proxy connection with Python

I have been attempting to connect to URLs from python. I have tried: urllib2, urlib3, and requests. It is the same issue that i run up against in all cases. Once I get the answer I imagine all three of …

Read More
Using pip behind a proxy with CNTLM

I am trying to use pip behind a proxy at work. One of the answers from this post suggested using CNTLM. I installed and configured it per this other post, but running cntlm.exe -c cntlm.ini -I -M http://…

Read More
How to get pip to work behind a proxy server

I am trying to use python package manager pip to install a package and it's dependencies from the internet. However I am behind a proxy in my college and have already set the http_proxy environment variable. But when I …

Read More