Use "byte-like object" from urlopen.read with JSON?

python python-3.x urlopen
r123454321 picture r123454321 · Jun 1, 2012 · Viewed 20.7k times · Source

Just trying to test out very simple Python JSON commands, but I'm having some trouble.

urlopen('http://www.similarsitesearch.com/api/similar/ebay.com').read()

should output

'{"num":20,"status":"ok","r0":"http:\\/\\/www.propertyroom.com\\/","r1":"http:\\/\\/www.ubid.com\\/","r2":"http:\\/\\/www.bidcactus.com\\/","r3":"http:\\/\\/www.etsy.com\\/","r4":"http:\\/\\/us.ebid.net\\/","r5":"http:\\/\\/www.bidrivals.com\\/","r6":"http:\\/\\/www.ioffer.com\\/","r7":"http:\\/\\/www.shopgoodwill.com\\/","r8":"http:\\/\\/www.beezid.com\\/","r9":"http:\\/\\/www.webidz.com\\/","r10":"http:\\/\\/www.auctionzip.com\\/","r11":"http:\\/\\/www.overstock.com\\/","r12":"http:\\/\\/www.bidspotter.com\\/","r13":"http:\\/\\/www.paypal.com\\/","r14":"http:\\/\\/www.ha.com\\/","r15":"http:\\/\\/www.onlineauction.com\\/","r16":"http:\\/\\/bidz.com\\/","r17":"http:\\/\\/www.epier.com\\/","r18":"http:\\/\\/www.sell.com\\/","r19":"http:\\/\\/www.rasmus.com\\/"}'

but I get that same string, with a b in front:

b'{"num":20,"status":"ok","r0":"http:\\/\\/www.propertyroom.com\\/","r1":"http:\\/\\/www.ubid.com\\/","r2":"http:\\/\\/www.bidcactus.com\\/","r3":"http:\\/\\/www.etsy.com\\/","r4":"http:\\/\\/us.ebid.net\\/","r5":"http:\\/\\/www.bidrivals.com\\/","r6":"http:\\/\\/www.ioffer.com\\/","r7":"http:\\/\\/www.shopgoodwill.com\\/","r8":"http:\\/\\/www.beezid.com\\/","r9":"http:\\/\\/www.webidz.com\\/","r10":"http:\\/\\/www.auctionzip.com\\/","r11":"http:\\/\\/www.overstock.com\\/","r12":"http:\\/\\/www.bidspotter.com\\/","r13":"http:\\/\\/www.paypal.com\\/","r14":"http:\\/\\/www.ha.com\\/","r15":"http:\\/\\/www.onlineauction.com\\/","r16":"http:\\/\\/bidz.com\\/","r17":"http:\\/\\/www.epier.com\\/","r18":"http:\\/\\/www.sell.com\\/","r19":"http:\\/\\/www.rasmus.com\\/"}'

Subsequently, when I try to run

json.loads(urlopen('http://similarsitesearch.com/api/similar/ebay.com').read())

it gives me the error message:

TypeError: can't use a string pattern on a bytes-like object"

which I'm assuming has something to do with the b?

I imported urlopen from urllib.request, and I am running Python 3.

Any ideas?

Answer

arviman picture arviman · Dec 12, 2012

The content from read() is of type bytes so you need to convert it to a string before trying to decode it into a json object.

To convert bytes to a string, change your code to: urlopen('http://similarsitesearch.com/api/similar/ebay.com').read().decode("utf-8")

Related questions

Let JSON object accept bytes or let urlopen output strings

With Python 3 I am requesting a json document from a URL. response = urllib.request.urlopen(request) The response object is a file-like object with read and readline methods. Normally a JSON object can be created with a file opened in …

Read More
Python 3.5.1 urllib has no attribute request

I have tried import urllib.request or import urllib The path for my urllib is /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/__init__.py I am wondering where is urlopen, or is my python module pointing to the wrong …

Read More
Again urllib.error.HTTPError: HTTP Error 400: Bad Request

Hy! I tried to open web-page, that is normally opening in browser, but python just swears and does not want to work. import urllib.request, urllib.error f = urllib.request.urlopen('http://www.booking.com/reviewlist.html?cc1=tr;pagename=…

Read More