json.loads with escape characters

Question

I'm trying to convert this string into python dict:

q = '{"request_body": "{\\x22username\\x22:\\x222\\x22,\\x22password\\x22:\\x226\\x22,\\x22id\\x22:\\x22e2cad174-736e-3041-cf7e\\x22, \\x22FName\\x22:\\x22HS\\x22}", "request_method": "POST"}'

when I do json.loads(a) it giving me an error simplejson.decoder.JSONDecodeError: Invalid \escape: line 1 column 19 (char 19) . So than I decided to convert \x22 into ". I followed suggestion from this question.

>>> q.decode('string_escape')
'{"request_body": "{"username":"2","password":"6","id":"e2cad174-736e-3041-cf7e", "FName":"HS"}", "request_method": "POST"}'
>>> 

But after that "{" part is invalid. So my question is what are the possible ways to convert string q into a python dict?

Answer 1

The problem is that JSON doesn't support the \xNN escape, only \uNNNN. Replacing \x22 with " will make the JSON invalid. What you should do is to replace \xNN with \u00NN:

>>> q = q.replace('\\x', '\\u00')
>>> json.loads(q)
{'request_body': '{"username":"2","password":"6","id":"e2cad174-736e-3041-cf7e", "FName":"HS"}', 'request_method': 'POST'}

(Note: This is not a complete solution as it will erratically replace correctly-escaped strings like abc\\xyz to abc\\u00yz. A complete solution probably needs a regex or even a lexer.)