NodeJS - How to stream request body without buffering

Question

In the below code I can't figure out why req.pipe(res) doesn't work, and yet doesn't throw an error either. A hunch tells me it's due to nodejs' asynch behavior, but this is a very simple case without a callback.

What am I missing?

http.createServer(function (req, res) {

  res.writeHead(200, { 'Content-Type': 'text/plain' });

  res.write('Echo service: \nUrl:  ' + req.url);
  res.write('\nHeaders:\n' + JSON.stringify(req.headers, true, 2));

  res.write('\nBody:\n'); 

  req.pipe(res); // does not work

  res.end();

}).listen(8000);

Here's the curl:

➜  ldap-auth-gateway git:(master) ✗ curl -v -X POST --data "test.payload" --header "Cookie:  token=12345678" --header "Content-Type:text/plain" localhost:9002 

Here's the debug output (see that body was uploaded):

  About to connect() to localhost port 9002 (#0)
  Trying 127.0.0.1...
    connected
    Connected to localhost (127.0.0.1) port 9002 (#0)
  POST / HTTP/1.1
  User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
  Host: localhost:9002
  Accept: */*
  Cookie:  token=12345678
  Content-Type:text/plain
  Content-Length: 243360
  Expect: 100-continue

  HTTP/1.1 100 Continue
  HTTP/1.1 200 OK
  Content-Type: text/plain
  Date: Sun, 04 Aug 2013 17:12:39 GMT
  Connection: keep-alive
  Transfer-Encoding: chunked

And the service responds without echoing the request body:

Echo service: 
Url:  /
Headers:
{
  "user-agent": "curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5",
  "host": "localhost:9002",
  "accept": "*/*",
  "cookie": "token=12345678",
  "content-type": "text/plain",
  "content-length": "243360",
  "expect": "100-continue"
}

... and final curl debug is

Body:
 Connection #0 to host localhost left intact
 Closing connection #0

Additionally, when I stress test with large request body, I get an EPIPE error. How can I avoid this?

-- EDIT: Through trial and error I did get this to work, and it still points to being a timing issue. Though it is still strange, as the timeout causes the payload to be returned, but the timeout duration is not minded. In other words whether I set the timeout to 5 seconds or 500 seconds, the payload is properly piped back to the request and the connection is terminated.

Here's the edit:

http.createServer(function (req, res) {

    try {
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.write('Echo service: ' + req.url + '\n' + JSON.stringify(req.headers, true, 2));
      res.write('\nBody:"\n');
      req.pipe(res);
    } catch(ex) {
      console.log(ex);
      // how to change response code to error here?  since headers have already been written?
    } finally {
      setTimeout((function() {
        res.end();
      }), 500000);
    }

}).listen(TARGET_SERVER.port);

?

Answer 1

Pipe req to res. Req is readable stream and response is a writable stream.It should work

   http.createServer(function (req, res) {

       res.writeHead(200, { 'Content-Type': 'text/plain' });    
       res.write('Echo service: ' + req.url + '\n' + JSON.stringify(req.headers, true, 2));

       // pipe request body directly into the response body
       req.pipe(res);       

   }).listen(9002);