I cannot for the life of me figure out how to perform an HTTP PUT request with verbatim binary data in Python 2.7 with the standard Python libraries.
I thought I could do it with urllib2, but that fails because urllib2.Request
expects its data in application/x-www-form-urlencoded
format. I do not want to encode the binary data, I just want to transmit it verbatim, after the headers that include
Content-Type: application/octet-stream
Content-Length: (whatever my binary data length is)
This seems so simple, but I keep going round in circles and can't seem to figure out how.
How can I do this? (aside from open up a raw binary socket and write to it)
I found out my problem. It seems there is some obscure behavior in urllib2.Request
/ urllib2.urlopen()
(at least in Python 2.7)
The urllib2.Request(url, data, headers)
constructor seems to expect the same type of string in its url and data parameters.
I was giving the data parameter raw data from a file read()
call (which in Python 2.7 returns it in the form of a 'plain' string), but my url was accidentally Unicode because I concatenated a portion of the URL from the result of another function which returned Unicode strings.
Rather than trying to "downcast" url
from Unicode -> plain strings, it tried to "upcast" the data
parameter to Unicode, and it gave me a codec error. (oddly enough, this happens on the urllib2.urlopen()
function call, not the urllib2.Request
constructor)
When I changed my function call to
# headers contains `{'Content-Type': 'application/octet-stream'}`
r = urllib2.Request(url.encode('utf-8'), data, headers)
it worked fine.