Choice of transports for JSON over TCP

fadedbee picture fadedbee · Jul 4, 2011 · Viewed 19.4k times · Source

I'm writing a simple streaming JSON service. It consists of JSON messages, sent intermittently, for a long period of time (weeks or months).

What is the best practise with regard to sending multiple JSON messages over a plain TCP socket?

Some alternatives I have looked at (and their downsides) are:

  1. newline separated JSON - downside: newlines within JSON require escaping, or prohibition
  2. websocket inspired 0x00 0xff framing - downside: it's now binary, not utf-8 anymore
  3. real websockets - downside: lack of (opensource) websocket client libraries
  4. http multipart http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html - downside: incomplete client support?
  5. no delimiters - downside: chunking requires JSON parsing (can't just count curlies because of curlies in strings)

Is there a good, or at least well-established way of doing this?

Answer

Javier picture Javier · Jul 4, 2011

my first two options would be:

  1. Do what early TCP protocols do: send one message (a JSON object in your case) and close the connection. The client detects it and reopens to get the next object.

    • pros: very easy to parse, no extra (content) bytes sent. any loss of data means losing just a single object. if you can stand that, there's no need to add retransmision to your app.
    • cons: if you send a (huge) lot of (very) small objects, the three-packet TCP handshake adds to latency.
  2. Do what chunked-mode HTTP does: first send the number of bytes in the JSON object, a newline (CRLF in HTTP), and your JSON object. The client just have to count bytes to know when the next byte would be the next objectsize.

    • pros: you keep one long-lived stream.
    • cons: a few extra bytes, you have to keep a long-lived stream, so accidental break and reconnection has to be handled as exceptional events, need to establish some handshaking to continue where it failed.