wait process until all subprocess finish?

python subprocess ipc
Nikhil Rupanawar picture Nikhil Rupanawar · Feb 27, 2013 · Viewed 173.5k times · Source

I have a main process which creates two or more sub processes, I want main process to wait until all sub processes finish their operations and exits?

 # main_script.py

 p1 = subprocess.Popen(['python script1.py']) 
 p2 = subprocess.Popen(['python script2.py'])
 ... 
 #wait main process until both p1, p2 finish
 ...

Answer

glglgl picture glglgl · Feb 27, 2013

A Popen object has a .wait() method exactly defined for this: to wait for the completion of a given subprocess (and, besides, for retuning its exit status).

If you use this method, you'll prevent that the process zombies are lying around for too long.

(Alternatively, you can use subprocess.call() or subprocess.check_call() for calling and waiting. If you don't need IO with the process, that might be enough. But probably this is not an option, because your if the two subprocesses seem to be supposed to run in parallel, which they won't with (check_)call().)

If you have several subprocesses to wait for, you can do

exit_codes = [p.wait() for p in p1, p2]

which returns as soon as all subprocesses have finished. You then have a list of return codes which you maybe can evaluate.

Related questions

blocks - send input to python subprocess pipeline

I'm testing subprocesses pipelines with python. I'm aware that I can do what the programs below do in python directly, but that's not the point. I just want to test the pipeline so I know how to use it. My …

Read More
How to call an external command?

How do you call an external command (as if I'd typed it at the Unix shell or Windows command prompt) from within a Python script?

Read More
Running shell command and capturing the output

I want to write a function that will execute a shell command and return its output as a string, no matter, is it an error or success message. I just want to get the same result that I would have …

Read More