Question: Is there a way to use flush=True
for the print()
function without getting the BrokenPipeError
?
I have a script pipe.py
:
for i in range(4000):
print(i)
I call it like this from a Unix command line:
python3 pipe.py | head -n3000
And it returns:
0
1
2
So does this script:
import sys
for i in range(4000):
print(i)
sys.stdout.flush()
However, when I run this script and pipe it to head -n3000
:
for i in range(4000):
print(i, flush=True)
Then I get this error:
print(i, flush=True)
BrokenPipeError: [Errno 32] Broken pipe
Exception BrokenPipeError: BrokenPipeError(32, 'Broken pipe') in <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'> ignored
I have also tried the solution below, but I still get the BrokenPipeError
:
import sys
for i in range(4000):
try:
print(i, flush=True)
except BrokenPipeError:
sys.exit()
The BrokenPipeError
is normal as said phantom because the reading process (head) terminates and closes its end of the pipe while the writing process (python) still tries to write.
Is is an abnormal condition, and the python scripts receives a BrokenPipeError
- more exactly, the Python interpreter receives a system SIGPIPE signal that it catches and raises the BrokenPipeError
to allow the script to process the error.
And you effectively can process the error, because in your last example, you only see a message saying that the exception was ignored - ok it is not true, but seems related to this open issue in Python : Python developpers think important to warn user of the abnormal condition.
What really happens is that AFAIK the python interpreter always signals this on stderr, even if you catch the exception. But you just have to close stderr before exiting to get rid of the message.
I slightly changed your script to :
Here is the script I used :
import sys
try:
for i in range(4000):
print(i, flush=True)
except (BrokenPipeError, IOError):
print ('BrokenPipeError caught', file = sys.stderr)
print ('Done', file=sys.stderr)
sys.stderr.close()
and here the result of python3.3 pipe.py | head -10
:
0
1
2
3
4
5
6
7
8
9
BrokenPipeError caught
Done
If you do not want the extraneous messages just use :
import sys
try:
for i in range(4000):
print(i, flush=True)
except (BrokenPipeError, IOError):
pass
sys.stderr.close()