Python3: JSON POST Request WITHOUT requests library

moritzg picture moritzg · Aug 25, 2014 · Viewed 41.2k times · Source

I want to send JSON encoded data to a server using only native Python libraries. I love requests but I simply can't use it because I can't use it on the machine which runs the script. I need to do it without.

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = urllib.parse.urlencode(newConditions)
params = params.encode('utf-8')

req = urllib.request.Request(conditionsSetURL, data=params)
urllib.request.urlopen(req)        

My server is a local WAMP server. I always get an

urllib.error.HTTPError: HTTP Error 500: Internal Server Error

I am 100% sure that this is NOT a server issue, because the same data, with the same url, on the same machine, with the same server works with the requests library and Postman.

Answer

Martijn Pieters picture Martijn Pieters · Aug 25, 2014

You are not posting JSON, you are posting a application/x-www-form-urlencoded request.

Encode to JSON and set the right headers:

import json

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = json.dumps(newConditions).encode('utf8')
req = urllib.request.Request(conditionsSetURL, data=params,
                             headers={'content-type': 'application/json'})
response = urllib.request.urlopen(req)

Demo:

>>> import json
>>> import urllib.request
>>> conditionsSetURL = 'http://httpbin.org/post'
>>> newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
>>> params = json.dumps(newConditions).encode('utf8')
>>> req = urllib.request.Request(conditionsSetURL, data=params,
...                              headers={'content-type': 'application/json'})
>>> response = urllib.request.urlopen(req)
>>> print(response.read().decode('utf8'))
{
  "args": {}, 
  "data": "{\"con4\": 40, \"con2\": 20, \"con1\": 40, \"password\": \"1234\", \"con3\": 99}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Connection": "close", 
    "Content-Length": "68", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "Python-urllib/3.4", 
    "X-Request-Id": "411fbb7c-1aa0-457e-95f9-1af15b77c2d8"
  }, 
  "json": {
    "con1": 40, 
    "con2": 20, 
    "con3": 99, 
    "con4": 40, 
    "password": "1234"
  }, 
  "origin": "84.92.98.170", 
  "url": "http://httpbin.org/post"
}