I'm porting ebay sdk to python3 and I've stumbled upon the following issue.
I'm using pycurl to send some HTTP requests. Here is how I configure it:
self._curl = pycurl.Curl()
self._curl.setopt(pycurl.FOLLOWLOCATION, 1)
self._curl.setopt(pycurl.URL, str(request_url))
self._curl.setopt(pycurl.SSL_VERIFYPEER, 0)
self._response_header = io.StringIO()
self._response_body = io.StringIO()
self._curl.setopt(pycurl.CONNECTTIMEOUT, self.timeout)
self._curl.setopt(pycurl.TIMEOUT, self.timeout)
self._curl.setopt(pycurl.HEADERFUNCTION, self._response_header.write)
self._curl.setopt(pycurl.WRITEFUNCTION, self._response_body.write)
When I call self._curl.perform() I get the following error:
pycurl.error: (23, 'Failed writing body (1457 != 1460)')
As far as I know this means that there is an issue with the write function, but I can't figure out what it is exactly. Could be related to migration from StringIO module to io, but I'm not sure.
UPD: I've tried the following:
def body(buf):
self._response_body.write(buf)
def header(buf):
self._response_header.write(buf)
self._curl.setopt(pycurl.HEADERFUNCTION, header)
self._curl.setopt(pycurl.WRITEFUNCTION, body)
and it works. I've tried to do the same trick with lambdas (instead of defining those awkward function, but it didn't work.
I believe the problem is that pycurl no longer functions with StringIO like desired. A solution is to use io.BytesIO instead. You can then get information written into the buffer and decode it into a string.
Using BytesIO with pycurl instead of StringIO:
e = io.BytesIO()
c.setopt(pycurl.WRITEFUNCTION, e.write)
Decoding byte information from the BytesIO object:
htmlString = e.getvalue().decode('UTF-8')
You can use any type of decoding you want, but this should give you a string object you can parse.
Hope this helps people using Python 3.