How to close urllib2 connection?

nobody picture nobody · Jul 6, 2011 · Viewed 23.8k times · Source

I have made a program using urllib2 that makes a lot of connections across the web. I noticed that eventually that this can be DDoS worthy; I would like to know how to close down each connection after I have done my business to prevent such an attack.

The code I am using to open a connection is:

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
r = opener.open("http://www.python.org)
html = r.read()

Answer

Blair picture Blair · Jul 6, 2011

I assume you are opening them with the urlopen() function. Its documentation states:

This function returns a file-like object with two additional methods:

As a file-like object, it will have a close method which you can call:

connection = urllib2.urlopen(url)
# Do cool stuff in here.
connection.close()

Update: Using the code you added to your question:

>>> import urllib2
>>> import cookielib
>>> cj = cookielib.CookieJar()
>>> opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
>>> r = opener.open("http://www.python.org")
>>> html = r.read()
>>> r.close??
Type:  instancemethod
Base Class: <type 'instancemethod'>
String Form: <bound method addinfourl.close of <addinfourl at 150857644 whose fp = <socket._fileobject object at 0x8fd48ec>>>
Namespace: Interactive
File:  /usr/lib/python2.6/urllib.py
Definition: r.close(self)
Source:
    def close(self):
        self.read = None
        self.readline = None
        self.readlines = None
        self.fileno = None
        if self.fp: self.fp.close()
        self.fp = None

So the close() method exists and actually does something:

>>> r.close()
>>> r.read()
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
TypeError: 'NoneType' object is not callable