Troubleshooting JVM CPU spikes
We're seeing an interesting (though quite severe) issue with one of our application servers: at a certain point in time, the CPU usage of the JVM running our web applications starts rising and keeps on rising until the applications eventually slow down to a crawl. The only way to fix it is to restart the Application server software.
- Application server: Spring tc Server (as the servers are hosted elsewhere, I currently do not know the exact version)
- Applications: relatively standard Spring 3 web applications (we do use an in-JVM EHCache though)
This brings me to a simple question; what can we do to troubleshoot this?
I have considered using VisualVM (or some other JVM monitoring tool), but the best they can do - in this particular case - is give me a thread dump, which still will not tell me what is eating up all the CPU time (unless I'm missing something).
Answer
You need to find out what it is doing? A common cause of this problem is running low on free memory. If this is not the cause, a CPU profiler is needed. VisualVM comes free with the JDK and can do both for you.
only you can't profile the application all the time
When this is happening you can do an ad hoc profiling by calling jstack multiple times a few second apart. You can use diff of the stack traces to help you can find the thread which are likely to be busy and consuming CPU.