How do I disable 'Transfer-Encoding: chunked' encoding in Varnish?

Question

Using Varnish 4, I have a set of backends that're responding with a valid Content-Length header and no Transfer-Encoding header.

On the first hit from a client, rather than responding to the client with those headers, Varnish is dropping the Content-Length header and adding Transfer-Encoding: chunked to the response. (Interestingly, the payload doesn't appear to have any chunks in it - it's one contiguous payload).

This causes serious problems for clients like Flash video players that are trying to do segment-size, bandwidth, etc analysis based on the Content-Length header. Their analysis fails, and they can't do things like multi-bitrate streaming, etc.

I've tried a number of semi-obvious things like:

• beresp.do_stream = true
• beresp.do_gzip = false
• unset req.http.Accept-Encoding

Sample backend response:

HTTP/1.1 200 OK
Cache-Control: public, max-age=600
Content-Type: video/mp4
Date: Tue, 13 May 2014 19:44:35 GMT
Server: Apache
Content-Length: 796618
Connection: keep-alive

Sample varnish response:

HTTP/1.1 200 OK
Server: Apache
Cache-Control: public, max-age=600
Content-Type: video/mp4
Date: Tue, 13 May 2014 23:10:06 GMT
X-Varnish: 2
Age: 0
Transfer-Encoding: chunked
Accept-Ranges: bytes

Subsequent loads of the object do including the Content-Length header, just not the first load into cache.

VCL: https://gist.github.com/onethumb/e64a405cc579909cace1

varnishlog output: https://gist.github.com/onethumb/e66a2bc4727a3a5340b6

Varnish Trac: https://www.varnish-cache.org/trac/ticket/1506

Answer 1

For the time being, do_stream = false will do what you want.

Avoiding chunked encoding for the case where the backend sends unchunked is a possible future improvement to Varnish.

Example:

sub vcl_backend_response {
        if(beresp.http.Content-Type ~ "video") {
                set beresp.do_stream = false;
                set beresp.do_gzip = false;
                //set resp.http.Content-Length = beresp.http.Content-Length;
        }
        if(beresp.http.Edge-Control == "no-store") {
                set beresp.uncacheable = true;
                set beresp.ttl = 60s;
                set beresp.http.Smug-Cacheable = "No";
                return(deliver);
        }
}