Last night all severs running Windows 2003 R2, IIS6 and ColdFusion/ASP.Net were sabotaged! Actually, any windows based web application making use of the IIS6 application pool was affected and guess who killed them… Havoc is consuming the inter-webs while non-technical users are asking the question; “who killed my web site”?

The answer, It was the boys over in Redmond, WA. Quite the debacle ( Fail Whale ) for Microsoft.

After installation of the following updates:

• Security Update for Windows Server 2003 (KB974318)
• Windows Malicious Software Removal Tool – December 2009 (KB890830)
• Security Update for Windows Server 2003 (KB973904)
• Update for Windows Server 2003 (KB971737)
• Update for Windows Server 2003 (KB973917)
• Security Update for Windows Server 2003 (KB974392)
• Cumulative Security Update for Internet Explorer 8 for Windows Server 2003 (KB976325)

We noticed multiple occurrences  the following warnings in the System event viewer:

A process serving application pool ‘DefaultAppPool’ terminated unexpectedly. The process id was ‘3524′. The process exit code was ‘0xffffffff’.

These warnings were immediately followed by a lovely error:

Application pool ‘DefaultAppPool’ is being automatically disabled due to a series of failures in the process(es) serving that application pool.

With these messages at hand it became apparent that one of the Windows Security Updates was to blame. Looking into the specifics of each of the updates I had conluded that the suspect was “Update for Windows Server 2003 (KB973917)“.

While I am still working to identify possible solutions for affected servers independently, removing Update 973917 has restored service temporarily for a number of my clients.

Anyone else see the Humor in this?

Take a lesson from this; Windows Automatic Update is kind of like a friend who secretly wants to cause you problems, because its fun….

Updated on December, 9 2009 at 1:59pm

The official response… get this; Microsoft wants us to reinstall Service Pack 2 for Windows Server 2003 on the affected installations to bring all IIS6.0 components up to date. Seems like a Cop out to me. Sure it would fix the issue but this is going to cause a lot of man hours for a number of companies.

Sure you can protect against SQL injection attacks at the code layer, but what happens when hackers find something you missed? With Rewrite Rules, using mod_rewrite on apache or ISAPI_Rewrite for IIS, you can add rules to ignore URL based SQL Injection all together. Below is and example of some basic protection you can add.

In ISAPI_Rewrite

RewriteRule .*DECLARE.* /security-violation.htm [I]
RewriteRule .*NVARCHAR.* /security-violation.htm [I]
RewriteRule .*INSERT .* /security-violation.htm [I]
RewriteRule .*INSERT %20.* /security-violation.htm [I]
RewriteRule .* xp_.* /security-violation.htm [I]
RewriteRule .*%20xp_.* /security-violation.htm [I]
RewriteRule .*%20@.* /security-violation.htm [I]
RewriteRule .* @.* /security-violation.htm [I]
RewriteRule .*@%20.* /security-violation.htm [I]
RewriteRule .*@ .* /security-violation.htm [I]
RewriteRule .*';* /security-violation.htm [I]
RewriteRule .*EXEC\(@.* /security-violation.htm [I]
RewriteRule .*sp_password.* /security-violation.htm [I]
RewriteRule /security-violation.htm /security.cfm[I,L]

In mod_rewrite

RewriteRule .*DECLARE.* /security-violation.htm [NC]
RewriteRule .*NVARCHAR.* /security-violation.htm [NC]
RewriteRule .*INSERT .* /security-violation.htm [NC]
RewriteRule .*INSERT %20.* /security-violation.htm [NC]
RewriteRule .* xp_.* /security-violation.htm [NC]
RewriteRule .*%20xp_.* /security-violation.htm [NC]
RewriteRule .*%20@.* /security-violation.htm [NC]
RewriteRule .* @.* /security-violation.htm [NC]
RewriteRule .*@%20.* /security-violation.htm [NC]
RewriteRule .*@ .* /security-violation.htm [NC]
RewriteRule .*';* /security-violation.htm [NC]
RewriteRule .*EXEC\(@.* /security-violation.htm [NC]
RewriteRule .*sp_password.* /security-violation.htm [NC]
RewriteRule /security-violation.htm /security.cfm[NC,L]

Add security.cfm to your webroot:

<cfoutput>
 
<h1>
HACK ATTEMPT LOGGED FROM IP: #remote_addr# 
</h1>
 
#DateFormat(Now(), "MM-DD-YYYY")# @ #TimeFormat(Now(), "HH:MM:SS")#
 
#script_name#&#query_string#
 
</cfoutput>
 
<cfmail to="Sysadmin" from="Your Website" subject="HACK ATTEMPT FROM IP: #remote_addr#">
 
HACK ATTEMPT RECORDED:
 
#DateFormat(Now(), "MM-DD-YYYY")# @ #TimeFormat(Now(), "HH:MM:SS")#
 
IP: #remote_addr#
 
ATTEMPT:
 
http://#server_name##script_name#&#query_string#
</cfmail>
<cfabort>