Profiling a python program with PyCharm (or any other IDE)

python pycharm profiler
Nickpick picture Nickpick · Oct 3, 2015 · Viewed 36.6k times · Source

I'm running a relatively complex python program and in it there is a montecarlo simulation which takes up most of the time. I would like to find out what part of it uses the most resources so I can potentially make it faster.

I'm using PyCharm Professional edition and tried to use the profiler, but the result is only a large list of irrelevant functions that I've never heard of.

Questions: Is there a good profiler I can use that delivers meaningful results so I can see which function or keyword uses the most resources in my montecarlo simulation?

Answer

shafeen picture shafeen · Oct 3, 2015

Depending on your needs and your python version, maybe you want to use something like hotshot. https://docs.python.org/2/library/hotshot.html

EDIT:

For python 3.4 cProfile is probably one the best options you have available but you will definitely have to filter the results with grep/sed/awk to be able to get the relevant results especially if you use libraries imported where there are a lot of internal calls occurring.

I like sorting by number of calls: python -m cProfile -s 'calls' <your_program>.py

Now the issue in python3 with that method is the number of primitive calls that will show up if cProfile is called externally, so running it internally is probably a better idea:

import cProfile

pr = cProfile.Profile()
pr.enable()
your_function_call()
pr.disable()
# after your program ends
pr.print_stats(sort="calls")

Related questions

How do I use installed packages in PyCharm?

In PyCharm, I've added the Python environment /usr/bin/python. However, from gnuradio import gr fails as an undefined reference. However, it works fine in the Python interpreter from the command line. GNURadio works fine with python outside of Pycharm. …

Read More
Unresolved reference issue in PyCharm

I have a directory structure ├── simulate.py ├── src │   ├── networkAlgorithm.py │   ├── ... And I can access the network module with sys.path.insert(). import sys import os.path sys.path.insert(0, "./src") from networkAlgorithm import * However, pycharm complains that it cannot access …

Read More
Trying to get PyCharm to work, keep getting "No Python interpreter selected"

I'm trying to learn Python and decided to use PyCharm. When I try to start a new project I get a dialog that says "No Python interpreter selected". It has a drop down to select a interpreter, but the drop …

Read More