How can I find memory leaks in my Python program?

python debugging memory-leaks twisted
Dickon Reed picture Dickon Reed · Feb 10, 2009 · Viewed 29.7k times · Source

Possible Duplicate:
Python memory profiler

I've got a fairly complex (about 20,000) line Python program which after some development has started consuming increasing amounts of memory when it runs. What are the best tools and techniques for finding out what all the memory is being used for?

Usually this comes down to either unexpectedly keeping references to objects, or extension module bugs (which isn't particularly likely since we've been using the Python 2.4 installation for a while).

We are using various third party libraries such as Twisted, Twisted Conch and MySQLdb.

Answer

S.Lott picture S.Lott · Feb 10, 2009

Generally, failing to close cursors is one of the most common kinds of memory leaks. The garbage collector can't see the MySQL resources involved in the cursor. MySQL doesn't know that the Python side was released unless the close() method is called explicitly.

Rule of thumb. Open, use and close cursors in as short a span of code as you can manage.

Related questions

Python memory leaks

I have a long-running script which, if let to run long enough, will consume all the memory on my system. Without going into details about the script, I have two questions: Are there any "Best Practices" to follow, which will …

Read More
Is there a built-in function to print all the current properties and values of an object?

So what I'm looking for here is something like PHP's print_r function. This is so I can debug my scripts by seeing what's the state of the object in question.

Read More
How do I run a Python program?

So I'm starting like Python a bit, but I'm having trouble erm...running it. Lol I'm using IDLE for now, but its no use whatsoever because you can only run a couple of lines at a time. I'm also using …

Read More