ValueError: unknown url type in urllib2, though the url is fine if opened in a browser

python urllib2 httprequest
Nik picture Nik · Apr 28, 2011 · Viewed 41.9k times · Source

Basically, I am trying to download a URL using urllib2 in python.

the code is the following:

import urllib2
req = urllib2.Request('www.tattoo-cover.co.uk')
req.add_header('User-agent','Mozilla/5.0')
result = urllib2.urlopen(req)

it outputs ValueError and the program crushes for the URL in the example. When I access the url in a browser, it works fine.

Any ideas how to handle the problem?

UPDATE:

thanks for Ben James and sth the problem is detected => add 'http://'

Now the question is refined: Is it possible to handle such cases automatically with some builtin function or I have to do error handling with subsequent string concatenation?

Answer

Ben James picture Ben James · Apr 28, 2011

When you enter a URL in a browser without the protocol, it defaults to HTTP. urllib2 won't make that assumption for you; you need to prefix it with http://.

Related questions

A good way to get the charset/encoding of an HTTP response in Python

Looking for an easy way to get the charset/encoding information of an HTTP response using Python urllib2, or any other Python library. >>> url = 'http://some.url.value' >>> request = urllib2.Request(url) >>&…

Read More
Import error: No module name urllib2

Here's my code: import urllib2.request response = urllib2.urlopen("http://www.google.com") html = response.read() print(html) Any help?

Read More
How to download image using requests

I'm trying to download and save an image from the web using python's requests module. Here is the (working) code I used: img = urllib2.urlopen(settings.STATICMAP_URL.format(**data)) with open(path, 'w') as f: f.write(img.read()) …

Read More