A process serving application pool 'X' suffered a fatal communication error with the Windows Process Activation Service
I'm running an ASP.NET 4.0 application under IIS 7.5. Several times a day, our application pool is recycling unexpectedly. Our system log shows the following event when this occurs:
A process serving application pool 'X' suffered a fatal communication error with the Windows Process Activation Service. The process id was '5768'. The data field contains the error number.
OR
A process serving application pool 'X' failed to respond to a ping. The process id was '1032'.
Almost always, there is a corresponding event at the exact same time in the Application log like this:
Faulting application name: w3wp.exe, version: 7.5.7600.16385, time stamp: 0x4a5bcd2b
Faulting module name: clr.dll, version: 4.0.30319.269, time stamp: 0x4ee9ae83
Exception code: 0xc00000fd
Fault offset: 0x00001916
Faulting process id: 0x508
Faulting application start time: 0x01cd4d8958ecf9ad
Faulting application path: C:\Windows\SysWOW64\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Report Id: 8dcc413b-b98a-11e1-8075-001c23d6d910
So I installed IIS Debugging Tools and set up a crash rule, specified "A specific IIS web application pool", no logging for first chance exceptions, and nothing under "Advanced Settings" (exceptions, breakpoints, events).
I'm also (concurrently) running ADPlus from the Windows Debugging Tools with the following command line:
adplus -crash -pn w3wp.exe -NoDumpOnFirst -o c:\logs
However, since I've attached the debuggers, I've gotten several "warning" level messages in the System log (the ones regarding the "Process serving application pool 'X' suffered a fatal communication error..."), but I am NOT getting the corresponding Error level event in the Application log anymore.
The only thing I've been able to get is the following (there were about 50 of these):
[6/18/2012 7:50:25 PM] Thread exited. Exiting thread system id - System ID: 3300. Exit code - 0x800703e9
[6/18/2012 7:50:25 PM] Thread exited. Exiting thread system id - System ID: 4992. Exit code - 0x800703e9
[6/18/2012 7:50:25 PM] Thread exited. Exiting thread system id - System ID: 5456. Exit code - 0x800703e9
[6/18/2012 7:50:25 PM] Thread exited. Exiting thread system id - System ID: 4924. Exit code - 0x800703e9
Exit code 0x800703e9 indicates a stack overflow somewhere, which is fortunate because that should be easy to fix once I can find it.
However, in order to do so, I need more information from my crash dumps so that I can open them with WinDbg and use the "!clrstack" command to pinpoint my problem.
My question is: Do I have the debugging tools configured incorrectly or am I misunderstanding the event logs? It seems our app pool is recycling whenever I see the event in the System log regarding the "fatal communication error with Windows Process Activation Service", however I'm not getting any crash dump info from IIS Debugging Tools or ADPlus when those events occur. And for some reason, since I've attached the debuggers, I'm not getting any more "Error" events in the Application log that correspond with my System log warning events. Not sure why. I somehow need to get the full crash dump with CLR information in it so I can figure out where my problem is.
The only other thing I can think to mention is that the Windows Error Reporting service is not running. Not sure if it needs to be or not.
Thanks for any help.
Answer
Both the exception code you were getting in the Application log, and thread exit code indicate you have stack overflow. It is puzzling why process is not crashing when thread exits with stack overflow error though. Anyways, to enable break on stack overflow exception, the command is:
sxe sov
Please note that you will need to use x86 version of the debugger in your case, because your application pool is configured as 32 bit process.