pass stdout as file name for command line util?

linux bash file-io stdout pipe
Jake picture Jake · Oct 13, 2011 · Viewed 30k times · Source

I'm working with a command line utility that requires passing the name of a file to write output to, e.g.

foo -o output.txt

The only thing it writes to stdout is a message that indicates that it ran successfully. I'd like to be able to pipe everything that is written to output.txt to another command line utility. My motivation is that output.txt will end up being a 40 GB file that I don't need to keep, and I'd rather pipe the streams than work on massive files in a stepwise manner.

Is there any way in this scenario to pipe the real output (i.e. output.txt) to another command? Can I somehow magically pass stdout as the file argument?

Answer

frankc picture frankc · Oct 13, 2011

Named pipes work fine, but you have a nicer, more direct syntax available via bash process substitution that has the added benefit of not using a permanent named pipe that must later be deleted (process substitution uses temporary named pipes behind the scenes):

foo -o >(other command)

Also, should you want to pipe the output to your command and also save the output to a file, you can do this:

foo -o >(tee output.txt) | other command

Related questions

How to redirect output to a file and stdout

In bash, calling foo would display any output from that command on the stdout. Calling foo > output would redirect any output from that command to the file specified (in this case 'output'). Is there a way to redirect output …

Read More
Split files using tar, gz, zip, or bzip2

I need to compress a large file of about 17-20 GB. I need to split it into several files of around 1GB per file. I searched for a solution via Google and found ways using split and cat commands. But …

Read More
Shell/Bash shortcut for bulk renaming of files in a folder

Is there a shortcut in Shell/Bash that can rename all the files in a folder based on a regex or some other criteria. What I am looking for here is in my folder documents, that has let's say a 100 …

Read More