python socket GET

python python-sockets
james smith picture james smith · Dec 10, 2015 · Viewed 42.5k times · Source

From the other posts on stack overflow this should be working

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)                 

s.connect(("www.cnn.com" , 80))
s.sendall("GET / HTTP/1.1\r\n")
print s.recv(4096)
s.close

but for some reason it just hangs (at recv) and never prints. I know that a request to www.cnn.com will chunk it's data but I should at least read something from recv, right?

p.s. I know this isn't the best way to do it and that there are library like httplib and urllib2 out there, but I can't use those for this project (it's for school). I have to use the socket library

Answer

Takis picture Takis · Dec 10, 2015

You forgot to send a blank line after your request line:

s.sendall("GET / HTTP/1.1\r\n\r\n")

Furthermore, HTTP 1.1 specifies you should add the Host header field as documented in the Host section in the HTTP 1.1 RFC.

s.sendall("GET / HTTP/1.1\r\nHost: www.cnn.com\r\n\r\n")

Related questions

Python requests speed up using keep-alive

In the HTTP protocol you can send many requests in one socket using keep-alive and then receive the response from server at once, so that will significantly speed up whole process. Is there any way to do this in python …

Read More
Difficulty using Python's socket.gethostbyaddr()

I am trying to reverse dns a list of IPs using socket.gethostbyaddr() in python, which returns 'Unknown Host' for some values, but using dig for the same ip returns the Hostname. Also, dig seems to be significantly faster than …

Read More
Check network connection from an IP address with Python

How can I check if there is still connection from a specific ip address using python.

Read More