How to pipe Node.js scripts together using the Unix | pipe (on the command line)?

Question

I see how to pipe stuff together using Node.js streams, but how do you pipe multiple scripts together using the Unix |, given that some of these scripts can be async?

$ ./a.js | ./b.js

Example:

a.js (chmod 0755)

#!/usr/bin/env node

setTimeout(function(){
  console.log(JSON.stringify({ foo: 'bar' }));
}, 10);

b.js (chmod 0755)

#!/usr/bin/env node

console.log(process.argv);

This is the output:

$ ./a.js | ./b.js
[ 'node', '/Users/viatropos/tests/b.js' ]

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: write EPIPE
    at errnoException (net.js:883:11)
    at Object.afterWrite (net.js:700:19)

At first glance it seems like there's a lot going wrong, so not really sure where to start. Is there a way to get this to work? The end goal is to be able to take the console.log output from ./a.js and use it in ./b.js. The reason is, most of the time these scripts will be run one at a time, but sometimes it would be nice to be able to pipe them together, so ideally the system should be able to handle both cases.

Answer 1

The problem is that your b.js immediately ends and closes its standard in, which causes an error in a.js, because its standard out got shut off and you didn't handle that possibility. You have two options: handle stdout closing in a.js or accept input in b.js.

Fixing a.js:

process.on("SIGPIPE", process.exit);

If you add that line, it'll just give up when there's no one reading its output anymore. There are probably better things to do on SIGPIPE depending on what your program is doing, but the key is to stop console.loging.

Fixing b.js:

#!/usr/bin/env node

var stdin = process.openStdin();

var data = "";

stdin.on('data', function(chunk) {
  data += chunk;
});

stdin.on('end', function() {
  console.log("DATA:\n" + data + "\nEND DATA");
});

Of course, you don't have to do anything with that data. They key is to have something that keeps the process running; if you're piping to it, stdin.on('data', fx) seems like a useful thing to do.

Remember, either one of those will prevent that error. I expect the second to be most useful if you're planning on piping between programs.