Is there a way to avoid undeployment memory leaks in Tomcat?
This question is for anyone who's ever tested the "Find leaks" button in the Tomcat manager and got some results like this:
The following web applications were stopped (reloaded, undeployed), but their classes from previous runs are still loaded in memory, thus causing a memory leak (use a profiler to confirm):
/leaky-app-name
I'm assuming this has something to do with that "Perm Gen space" error you often get with frequent redeployments.
So what I'm seeing in jconsole when I deploy is that my loaded classes goes from about 2k to 5k. Then you would think an undeployment should drop them back down to 2k but they remain at 5k.
I've also tried using the following JVM options:
-XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled
I did see VERY minor dips in the amount of Perm Gen space used but not what I expected and the loaded class counts did not drop.
So is there a way to configure Tomcat or design your app to unload better on an undeployment? Or are we stuck with restarting the server after some major debugging sessions?
Tomcat version output:
Server version: Apache Tomcat/6.0.29
Server built: July 19 2010 1458
Server number: 6.0.0.29
OS Name: Windows 7
OS Version: 6.1
Architecture: x86
JVM Version: 1.6.0_18-b07
JVM Vendor: Sun Microsystems Inc.
Update:
Thanks to celias' answer I decided to do a little more digging and I think I determined the culprit to be in my application thanks to CXF, Spring and JAXB.
After I learned how to profile a Java application, I pointed the profiler at Tomcat and took some heap dumps and snapshots to see what the objects and classes looked like in memory. I discovered that some of the enumerations from my XML schema used in my CXF/JAXB (wsdl2java) generated classes were lingering after an undeployment. According to my heap dump it looks like the objects were tied to a Map. Disclaimer: I admit I'm still a little green with profiling and tracing an object's call tree can be challenging in Java.
Also I should mention that I didn't even invoke the service, just deployed then undeployed it. The objects themselves appeared to be loaded via reflection initiated from Spring on deployment. I believe I followed the convention for setting up a CXF service in Spring. So I'm not 100% sure if this is Spring/CXF, JAXB, or reflection's fault.
As a side note: the application in question is a web service using Spring/CXF and the XML happens to be a rather complex schema (an extension of NIEM).
Answer
If you want to make sure not to cause leaks you have to do the following:
- Make sure your web application does not use any java classes that are in the web container shared libraries. If you have any shared libraries, make sure there is no strong references to the objects in those libraries
- Avoid using static variables, especially on java objects like HashTable, Sets, etc. If you need to, make sure that you call remove to release the objects with the maps, lists...
Here is also a good article on ThreadLocal and MemoryLeaks - http://blog.arendsen.net/index.php/2005/02/22/threadlocals-and-memory-leaks-revisited/