What causes "java.io.IOException: stream was reset: CANCEL" with okhttp and spdy?

Question

I'm experimenting with OKHttp (version 2.0.0-RC2) and SPDY and seeing IOException: stream was reset: CANCEL quite a lot, maybe 10% or more of all requests in some preliminary testing. When using Apache HttpClient and regular https we were not seeing any equivalent issue as far as I'm aware. I'm pretty sure we also don't see anything equivalent with OkHttp when SPDY is disabled (client.setProtocols(ImmutableList.of(Protocol.HTTP_1_1))) but I haven't done enough testing to be 100% confident.

This previous question sees these exceptions among others and the advice there is to ignore them, but this seems crazy: we get an exception while reading data from the server, so we abort the data processing code (which using Jackson). We need to do something in such cases. We could retry the request, of course, but sometimes it's a POST request which is not retry-able, and if we've already started receiving data from the server then it's a good bet that the server as already taken the requested action.

Ideally there is some configuration of the client and/or the server that we can do in order to reduce the incidence of these exceptions, but I don't understand SPDY well enough to know even where to start looking or to advise our server-admin team to start looking.

Stack trace, in case it's helpful:

java.io.IOException: stream was reset: CANCEL
  at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.checkNotClosed(SpdyStream.java:442)
  at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.read(SpdyStream.java:344)
  at com.squareup.okhttp.internal.http.SpdyTransport$SpdySource.read(SpdyTransport.java:273)
  at okio.RealBufferedSource.exhausted(RealBufferedSource.java:60)
  at okio.InflaterSource.refill(InflaterSource.java:96)
  at okio.InflaterSource.read(InflaterSource.java:62)
  at okio.GzipSource.read(GzipSource.java:80)
  at okio.RealBufferedSource$1.read(RealBufferedSource.java:227)
  at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.loadMore(UTF8StreamJsonParser.java:174)
  at com.fasterxml.jackson.core.base.ParserBase.loadMoreGuaranteed(ParserBase.java:431)
  at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2111)
  at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2092)
  at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:275)
  at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:205)
  at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:230)
  at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:202)
  at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
  at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
  at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2765)
  at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:1546)
  at com.fasterxml.jackson.core.JsonParser.readValueAsTree(JsonParser.java:1363)
  at (application-level code...)

Answer 1

Your best bet is to set a breakpoint in the two places where the CANCEL error code is assigned: that's SpdyStream#closeInternal (line 246) and SpdyStream#receiveRstStream (line 304). If you can put a breakpoint here, you can capture who is canceling your stream and that'll shed light on the problem.

If for whatever reason you cannot attach a debugger, you can instrument the code to print a stacktrace when those lines are reached:

new Exception("SETTING ERROR CODE TO " + errorCode).printStackTrace();

In either case, I'm the author of that code and I'd love to help you resolve this problem.