IIS Service Unavailable aka. Windows Security update kills application pool
Last night all severs running Windows 2003 R2, IIS6 and ColdFusion/ASP.Net were sabotaged! Actually, any windows based web application making use of the IIS6 application pool was affected and guess who killed them… Havoc is consuming the inter-webs while non-technical users are asking the question; “who killed my web site”?
The answer, It was the boys over in Redmond, WA. Quite the debacle ( Fail Whale ) for Microsoft.
After installation of the following updates:
- Security Update for Windows Server 2003 (KB974318)
- Windows Malicious Software Removal Tool – December 2009 (KB890830)
- Security Update for Windows Server 2003 (KB973904)
- Update for Windows Server 2003 (KB971737)
- Update for Windows Server 2003 (KB973917)
- Security Update for Windows Server 2003 (KB974392)
- Cumulative Security Update for Internet Explorer 8 for Windows Server 2003 (KB976325)
We noticed multiple occurrences the following warnings in the System event viewer:
A process serving application pool ‘DefaultAppPool’ terminated unexpectedly. The process id was ’3524′. The process exit code was ’0xffffffff’.
These warnings were immediately followed by a lovely error:
Application pool ‘DefaultAppPool’ is being automatically disabled due to a series of failures in the process(es) serving that application pool.
With these messages at hand it became apparent that one of the Windows Security Updates was to blame. Looking into the specifics of each of the updates I had conluded that the suspect was “Update for Windows Server 2003 (KB973917)“.
While I am still working to identify possible solutions for affected servers independently, removing Update 973917 has restored service temporarily for a number of my clients.
Anyone else see the Humor in this?
Take a lesson from this; Windows Automatic Update is kind of like a friend who secretly wants to cause you problems, because its fun….
Updated on December, 9 2009 at 1:59pm
The official response… get this; Microsoft wants us to reinstall Service Pack 2 for Windows Server 2003 on the affected installations to bring all IIS6.0 components up to date. Seems like a Cop out to me. Sure it would fix the issue but this is going to cause a lot of man hours for a number of companies.
Comments(16)
You are awesome. You saved me a day with this post. I uninstalled the updates you mention and my problems went away. I’m working on reinstalling SP2 right now and then I’ll reapply the updates. Hopefully all goes well.
This morning our intranet server had exactly the same error, I uninstalled kb973917 as specified in your post, rebooted and problems solved, thanks for the advise.
Same as buddy, this update b0rked 3 of my IIS machines!
Thanks a lot for this post.You saved me !! T
Thank you. I hope MS will give elucidations about!
Bye
Xanadu
Thanks for your timely post. I spent 4+ hours trying to troubleshoot this issue. This morning I decided to investigate any “malicious” patches that may have been installed by Microsoft. This post was at the top of Google. Uninstalled the patch and boom, back in business. Thanks!
Thank you very much for the post. This saved me a lot of time. Here is a similar response with some additional detail:
Previous to the installation of the KB973917 update, one or more of the core IIS .dll files were not at the correct file version. Specifically, the earlier installation of Windows Server 2003 Service Pack 2 on the server did not complete successfully, leaving some of the IIS .dlls at the Service Pack 1 level while bringing others up to the Service Pack 2 level. The IIS services had been able to run successfully even with that file mismatch in place. However, the installation of the KB973917 update exposes this pre-existing file mismatch environment to the degree that IIS is now unable to function properly.
To resolve this problem, reinstall Service Pack 2 for Windows Server 2003 on the web server. This will bring all IIS 6.0 components up to the correct file versions, and will maintain the installation of the KB973917 update. Reinstalling the KB973917 update should not be necessary.
Thanks mate, Spent a good 4hrs on this. Would have probably taken a few days to spot this otherwise.
You beauty, thanks very much!
Another thank you!! We had some prod report servers go down thanks to this patch you also saved me.. I owe you a beer!!
Packetboy.
THANKS! I can’t believe this! I uninstalled and then reinstalled SP2 and still had issues with permissions with several processes.
This was a huge mess.
A bigger problem is that Redmond didn’t correct the issue and the update was sent down AGAIN several days later!
You are the man! Thank you so much! I was this close to re-installing IIS…so nice to simply uninstall that patch so that our CRM Reporting features worked again!
I’m still having problems related to this update. Microsoft has been totally unresponsive. Uninstalling the security update above allows IIS to run without errors but some other software we run is still crippled and either does not run or terminates related processes.
Have you tried reinstalling Service Pack 2? Though inelegant, this should fix the issue for you.
Thanks a bunch for this. I just spent the last two hours trying to figure this out to no avail until I found this. (Would have discovered the problem sooner, but hadn’t worked on this machine since Nov 2009!)
Thanks again!
Damned…
You saved my life guys !!!
Simply uninstall this damned stuff…
And it wroked fine !!!
Bless you
Edouard