What is the difference between multiprocessing and subprocess?

python subprocess multiprocessing
Jun HU picture Jun HU · Nov 28, 2012 · Viewed 24.6k times · Source

My work should use parallel techniques, and I a new user of python. So I wonder if you could share some material about the python multiprocessing and subprocess modules. What is the difference between these two?

Answer

Martijn Pieters picture Martijn Pieters · Nov 28, 2012

The subprocess module lets you run and control other programs. Anything you can start with the command line on the computer, can be run and controlled with this module. Use this to integrate external programs into your Python code.

The multiprocessing module lets you divide tasks written in python over multiple processes to help improve performance. It provides an API very similar to the threading module; it provides methods to share data across the processes it creates, and makes the task of managing multiple processes to run Python code (much) easier. In other words, multiprocessing lets you take advantage of multiple processes to get your tasks done faster by executing code in parallel.

Related questions

How to spawn parallel child processes on a multi-processor system?

I have a Python script that I want to use as a controller to another Python script. I have a server with 64 processors, so want to spawn up to 64 child processes of this second Python script. The child script is …

Read More
How to call an external command?

How do you call an external command (as if I'd typed it at the Unix shell or Windows command prompt) from within a Python script?

Read More
Running shell command and capturing the output

I want to write a function that will execute a shell command and return its output as a string, no matter, is it an error or success message. I just want to get the same result that I would have …

Read More