java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bind
Tomcat is running a webapp under Windows. After a few days (under very low load), the exception mentioned in the title starts to appear in the logs, no new connections can be established from that point on, the only fix is then to reboot the server.
Environment:
- Latest Tomcat 6
- Windows Server 2008 R2
- JDK 6 update 30
- SQL Server 2008
- Kerberos authentication
Evidence collected so far:
- netstat shows no excessive amount of connections
- ProcessExplorer shows no excessive amount of open file handles
- system main memory usage is average
- JVM heap usage is average
- restarting Tomcat does not solve the problem
Open questions:
- if we were leaking connections, shouldn't they show up in netstat?
- shouldn't a restart of the appserver resolve the problem, because the OS should free all process resources?
- is there a way to trace the problem to its origin? E.g. installing monitoring software, maybe something similar to lsof etc.?
I'm out of ideas, any hints appreciated!
Answer
The reason we got this error is a bug in Windows Server 2008 R2 / Windows 7. The kernel leaks loopback sockets due to a race condition on machines with more than one core, this patch fixes the issue: http://support.microsoft.com/kb/2577795