os.kill not working on spawned process

python multiprocessing kill-process
Wes picture Wes · Feb 26, 2015 · Viewed 11.5k times · Source

I have written my own Python-based job scheduler which uses the multiprocessing module, to spawn new jobs. I'm trying to implement a feature to kill a running process using os.kill, but it is not working. My (simplified) code look like the following:

from multiprocessing import Process
import os
...
p = Process(target=self.start_job, args=(run_dir,cmd,))
p.start()
...
def start_job(self,run_dir,cmd):
        os.chdir(run_dir)
        os.system(cmd)
        print os.getpid()
        ...

I want to take this pid that is output (e.g. 3064) and from another python session run:

import os, signal    
os.kill(3064, signal.SIGTERM)

os.kill works if I run it on the pid of the parent/spawning process, but it does not work if I execute it on the pid of the child/spawned process. In addition to SIGTERM, I've also tried a number of other signals such as SIGKILL, SIGQUIT, etc. None of them worked either. Any help would be greatly appreciated.

Answer

Wes picture Wes · Aug 19, 2015

I solved this problem by creating a process group using the subprocess module to create the process and using os.killpg() to kill the process as described here: https://stackoverflow.com/a/4791612/2636544

Related questions

How to kill all Pool workers in multiprocess?

I want to stop all threads from a single worker. I have a thread pool with 10 workers: def myfunction(i): print(i) if (i == 20): sys.exit() p = multiprocessing.Pool(10, init_worker) for i in range(100): p.apply_async(myfunction, (i,)) …

Read More
Python multiprocessing: How to close the multiprocessing pool on exception

I am using python multiprocessing to split one of the longer processes and run parallelly. It is working fine except when there is an exception in one of the child processes, in which case, process pool is not closed and …

Read More
How to use multiprocessing pool.map with multiple arguments?

In the Python multiprocessing library, is there a variant of pool.map which supports multiple arguments? text = "test" def harvester(text, case): X = case[0] text+ str(X) if __name__ == '__main__': pool = multiprocessing.Pool(processes=6) case = RAW_DATASET pool.…

Read More