Why does Google Search return HTTP Error 403?
Consider the following Python code:
30 url = "http://www.google.com/search?hl=en&safe=off&q=Monkey" 31 url_object = urllib.request.urlopen(url); 32 print(url_object.read());
When this is run, an Exception is thrown:
File "/usr/local/lib/python3.0/urllib/request.py", line 485, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
However, when this is put into a browser, the search returns as expected. What's going on here? How can I overcome this so I can search Google programmatically?
Any thoughts?
Answer
this should do the trick
user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
url = "http://www.google.com/search?hl=en&safe=off&q=Monkey"
headers={'User-Agent':user_agent,}
request=urllib2.Request(url,None,headers) //The assembled request
response = urllib2.urlopen(request)
data = response.read() // The data u need