Tuning HTTP 400 and 500 rules in IIS MP

The witty and talented Marcus Oh posted a few remarks (on my birthday no less!) last year about the 'All 400 Errors' and 'All 500 Errors' (see iis management pack - useless rules ). As a former support engineer for a large e-commerce environment, this is a topic near and dear to my heart, and I have a couple of things to add to Marcus' very Trenchant Observation to explain WHY these rules are so noisy, and what can potentially be done about it short of disabling them entirely. The rules Marcus was referring to are: Microsoft Windows Internet Information Services\Internet Information Services 6.0\Core Services\World Wide Web Publishing Service All HTTP 500 Errors (incidentally, there is a matching rule in the same subfolder under IIS 5.0) All HTTP 400 Errors (ditto) The first is simply to say Marcus makes an excellent point, they generate a lot of event noise. And left to their own devices, they can cause serious database bloat.(In MOM 2000 I saw these rules drag an entire e-commerce web farm into the dirt! Danger Will Robinson, Danger!). As a rule, I create rule disable overrides on high-volume e-commerce sites, as there are many very expensive, advanced tools for monitoring these errors on big money sites in near real-time without the hit to system resources required by MOM to parse an IIS log that grows larger than 1gb per server / per day. Not a knock against MOM, just suggesting one should employ the best tool for the task at hand. The 2nd, is that someone did not do a very good job in allowing us to deal with the parameters in the IIS logs. The sc-substatus field in the IIS log (the .2 of a 401.2) is stored in parameter 27 (see IIS MP technical Reference for complete listing), but when we look at the fields available in Advanced Criteria of the rule properties, we can only filter up to parameter 25. This means the sc-substatus fields we wish to ignore cannot be easily ignored, and highlights that fact that the real fix for the situation is to make 26 and 27 accessible in the Console GUI. For now, let me provide some alternatives to disabling these rules altogether. Figure 1: Advanced Criteria Parameters All HTTP 500 Errors Rule: All HTTP 500 Errors The 'HTTP 500 Errors' indicates an 'Internal Server Error'. This can be indicative of real problems, so I don't disable this rule. HOWeVER, I do see that most of what I see from this error comes mostly during my nightly backup period when my Exchange OWA Server disk I/O is in utter gridlock (causing my OWA synthetic transaction to fail), so I simply configured the schedule on this rule to exclude my nightly backup window - problem solved....Ok, ok, problem slightly improved. But don't ignore these errors entirely by disabling the rule if you can bear it. All HTTP 400 Errors Rule: All HTTP 400 Errors This is a slightly more involved discussion. What you'll find with this one is that you'll see a truckload of 401.1 and 401.2 errors on sites that require authentication...But why? Some background on authentication with IE... When you use Internet Explorer to access a website that does not allow anonymous authentication, IE attempts access on your behalf in the following order: 1) Anonymous. (A 401.1 and 401.2 are generated when this fails) 2) Windows Integrated. If this fails, then 3) Basic. (you'll get a logon prompt. This passes credentials in clear text, so avoid this when you can) When you access a website, IE will still try to use Anonymous Access first, but when IIS sends a message that authentication is required, it will automatically attempt to authenticate (windows Integrated, then Basic). Note that this is not true if the website is in the Internet Zone. Unlike Basic authentication, Windows Integrated does not initially prompt users for a user name and password. And you're not passing credential in cleartext as is done with Basic authentication. Here's a clip from an IIS log of a hit against a website with anonymous turned off and Windows Integrated auth enabled 2006-04-23 18:17:57 W3SVC1 MON-MGT-02 192.168.243.62 GET /test/blank.htm - 80 - 192.168.243.1 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+2.0.50727;+.NET+CLR+1.1.4322) - - mon-mgt-02 401 2 2148074254 1872 420 150 2006-04-23 18:17:57 W3SVC1 MON-MGT-02 192.168.243.62 GET /test/blank.htm - 80 - 192.168.243.1 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+2.0.50727;+.NET+CLR+1.1.4322) - - mon-mgt-02 401 1 0 2204 504 15 2006-04-23 18:17:57 W3SVC1 MON-MGT-02 192.168.243.62 GET /test/blank.htm - 80 akos-corp\pz0003 192.168.243.1 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+2.0.50727;+.NET+CLR+1.1.4322) - - mon-mgt-02 200 0 0 1680 664 135 To break it down: The first attempt returns 401.2 - Logon failed due to server configuration, followed by 401.1 - Logon failed. And then finally we get HTTP status 200 - Success! Workarounds... 401.2 From the many logs I've parsed in researching this, it seems a 401.2 is always accompanied by a numeric NT Status Number (sc-win32-status in W3C terms) of '2148074254' in Parameter 18 (when the web server is IIS, anyway). So, to kill the events generated by 401.2 errors that contain this very common sc-win32-status value, we simply add the following criteria Field: Parameter 18 Condition: Equals Value:2148074254 This allows us to eliminate a large volume of these superfluous events, without ignoring 401.2 errors altogether. Now if I get a 401.2 alert, I know something unusual is afoot. 401.1 These dont have a convenient sc-win32-status, and frankly, I have not found a consistent value in other field to key on between Parameters 1 and 25. If anyone has any ideas, let me know. In reality, the vast majority of the 401 errors you'll see will be .1 or .2 (.3 - .7 are much less common). So if the 401.1's are still too many, as a last resort, create disable overrides for your high-volume web servers to prevent database bloat. All HTTP 400 Errors Rule Advanced Criteria in my MOM Installation