Python urllib2 with keep alive

python http urllib2 keep-alive
ibz picture ibz · Jun 24, 2009 · Viewed 36.5k times · Source

How can I make a "keep alive" HTTP request using Python's urllib2?

Answer

msanders picture msanders · Jun 24, 2009

Use the urlgrabber library. This includes an HTTP handler for urllib2 that supports HTTP 1.1 and keepalive:

>>> import urllib2
>>> from urlgrabber.keepalive import HTTPHandler
>>> keepalive_handler = HTTPHandler()
>>> opener = urllib2.build_opener(keepalive_handler)
>>> urllib2.install_opener(opener)
>>> 
>>> fo = urllib2.urlopen('http://www.python.org')

Note: you should use urlgrabber version 3.9.0 or earlier, as the keepalive module has been removed in version 3.9.1

There is a port of the keepalive module to Python 3.

Related questions

Using an HTTP PROXY - Python

I familiar with the fact that I should set the HTTP_RPOXY environment variable to the proxy address. Generally urllib works fine, the problem is dealing with urllib2. >>> urllib2.urlopen("http://www.google.com").read() returns urllib2.…

Read More
Python-Requests close http connection

I was wondering, how do you close a connection with Requests (python-requests.org)? With httplib it's HTTPConnection.close(), but how do I do the same with Requests? Code: r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={'track':…

Read More
catch specific HTTP error in python

I want to catch a specific http error and not any one of the entire family.. what I was trying to do is -- import urllib2 try: urllib2.urlopen("some url") except urllib2.HTTPError: <whatever> but what I …

Read More