Top "Multiprocessing" questions

Multiprocessing is the use of two or more central processing units (CPUs) within a single computer system.

Do multiprocessing pools give every process the same number of tasks, or are they assigned as available?

When you map an iterable to a multiprocessing.Pool are the iterations divided into a queue for each process in …

python multiprocessing pool
Applying Spacy Parser to Pandas DataFrame w/ Multiprocessing

Say I have a dataset, like iris = pd.DataFrame(sns.load_dataset('iris')) I can use Spacy and .apply to …

python nlp multiprocessing spacy
parallel post requests using multiprocessing and requests in Python

I have small code snippet as below: import requests import multiprocessing header = { 'X-Location': 'UNKNOWN', 'X-AppVersion': '2.20.0', 'X-UniqueId': '2397123', …

python-2.7 parallel-processing multiprocessing python-requests grequests
Doxygen is Slow

Doxygen takes about 12 hours to run on our code base. This is primarily because there is a lot of code …

multithreading performance multiprocessing doxygen
Sharing many queues among processes in Python

I am aware of multiprocessing.Manager() and how it can be used to create shared objects, in particular queues which …

python queue multiprocessing python-multiprocessing
PyInstaller-built Windows EXE fails with multiprocessing

In my project I'm using Python's multiprocessing library to create multiple processes in __main__. The project is being packaged into …

python windows python-2.7 multiprocessing pyinstaller
Python Process won't call atexit

I'm trying to use atexit in a Process, but unfortunately it doesn't seem to work. Here's some example code: import …

python multiprocessing terminate atexit
Using python's multiprocessing on slurm

I am trying to run some parallel code on slurm, where the different processes do not need to communicate. Naively …

python multiprocessing slurm
Create DB connection and maintain on multiple processes (multiprocessing)

Similar to another post I made, this answers that post and creates a new question. Recap: I need to update …

python database-connection multiprocessing
How to solve memory issues problems while multiprocessing using Pool.map()?

I have written the program (below) to: read a huge text file as pandas dataframe then groupby using a specific …

python pandas memory multiprocessing python-multiprocessing