twisted http client

python http client twisted
jldupont picture jldupont · Jan 27, 2010 · Viewed 12.6k times · Source

I am after an example describing the usage of Twisted's HTTP Client.

After reading the excellent blog post on the internals of Twisted, I understand how the "Factory" and "Protocol" components play their role but I am unclear on how to introduce "Request" in the overall Client flow.

More specifically, I need to be able to perform HTTP GET and POST requests to a remote server using Twisted.

Updated: after a discussion on irc #twisted / #python, it seems that twisted.web2 is fading away in favor of beefing up functionality on twisted.web e.g. Agent.

Answer

Jean-Paul Calderone picture Jean-Paul Calderone · Jan 27, 2010

As of Twisted 9.0, there are actually two HTTP clients available. The older one has quite a few features, such as automatically following redirects, interpreting cookie headers, etc. You can find an example of its usage here:

http://twistedmatrix.com/documents/current/web/examples/

(getpage.py and dlpage.py)

Unfortunately, the interface presented by the older client makes a number of common tasks difficult. For example, using getPage, you cannot examine arbitrary response headers.

The newer HTTP client isn't yet as featureful as the old one, but it presents an interface intended to eliminate the limitations of getPage. It is also intended to be more easily extended and customized. You can find a document describing its usage here:

http://twistedmatrix.com/documents/current/web/howto/client.html

Related questions

How do I download a file over HTTP using Python?

I have a small utility that I use to download an MP3 file from a website on a schedule and then builds/updates a podcast XML file which I've added to iTunes. The text processing that creates/updates the XML …

Read More
What is the quickest way to HTTP GET in Python?

What is the quickest way to HTTP GET in Python if I know the content will be a string? I am searching the documentation for a quick one-liner like: contents = url.get("http://example.com/foo/bar") But all I …

Read More
How do I send a POST request as a JSON?

data = { 'ids': [12, 3, 4, 5, 6 , ...] } urllib2.urlopen("http://abc.com/api/posts/create",urllib.urlencode(data)) I want to send a POST request, but one of the fields should be a list of numbers. How can I do that ? (JSON?)

Read More