Why does my Python program average only 33% CPU per process? How can I make Python use all available CPU?

Question

I use Python 2.5.4. My computer: CPU AMD Phenom X3 720BE, Mainboard 780G, 4GB RAM, Windows 7 32 bit.

I use Python threading but can not make every python.exe process consume 100% CPU. Why are they using only about 33-34% on average?.

I wish to direct all available computer resources toward these large calculations so as to complete them as quickly as possible.

EDIT: Thanks everybody. Now I'm using Parallel Python and everything works well. My CPU now always at 100%. Thanks all!

Answer 1

It appears that you have a 3-core CPU. If you want to use more than one CPU core in native Python code, you have to spawn multiple processes. (Two or more Python threads cannot run concurrently on different CPUs)

As R. Pate said, Python's multiprocessing module is one way. However, I would suggest looking at Parallel Python instead. It takes care of distributing tasks and message-passing. You can even run tasks on many separate computers with little change to your code.

Using it is quite simple:

import pp

def parallel_function(arg):
    return arg

job_server = pp.Server() 

# Define your jobs
job1 = job_server.submit(parallel_function, ("foo",))
job2 = job_server.submit(parallel_function, ("bar",))

# Compute and retrieve answers for the jobs.
print job1()
print job2()