Why am I getting "cat: write error: Broken pipe" rarely and not always

linux bash grep cat broken-pipe
Vishal-Lia picture Vishal-Lia · Apr 5, 2018 · Viewed 17.6k times · Source

I am running some scripts with commands having cat pipelined with grep like:

cat file.txt | grep "pattern"

Most of the times there are no problems. But sometimes I get below error:

cat: write error: Broken pipe

So how do I find out when the command is causing this problem and why?

Answer

Inian picture Inian · Apr 5, 2018

The reason is because the pipe is closed by grep when it still has some data to be read from cat. The signal SIGPIPE is caught by cat and it exits.

What usually happens in a pipeline is the shell runs cat in one process and grep in another. The stdout of cat is connected to the write-end of the pipe and stdin of grep to the read end. What happened was grep hit a pattern search that did not exist and exited immediately causing the read end of the pipe to be closed, which cat does not like since it has some more data to be write out to the pipe. Since the write actions happens to an other which has been closed other end, SIGPIPE is caught by the cat on which it immediately exits.

For such a trivial case, you could remove the pipeline usage altogether and run it as grep "pattern" file.txt when the file's contents are made available over the stdin of grep on which it could read from.

Related questions

Recursive cat all the files into single file

I have bunch of files sitting in folders like data\A\A\A\json1.json data\A\A\A\json2.json data\A\A\B\json1.json ... data\Z\Z\Z\json_x.json I want to cat all the …

Read More
Grep all instances of strings that start with certain characters

I would like to grep out all instances of strings that start with the characters 'rs' (from just one file) and pipe the full string into a new file. I managed to get the count of the instances but I …

Read More
How to get the process ID to kill a nohup process?

I'm running a nohup process on the server. When I try to kill it my putty console closes instead. this is how I try to find the process ID: ps -ef |grep nohup this is the command to kill kill …

Read More