October 19, 2004

IIS vs Apache Defects

Michael posted an interesting article comparing the defects of IIS6 against those of Apache 2.

The results? See for yourself:


Michael followed up with a second post, taking care of 4 major comments from people who saw the original post, which included:

  1. Perhaps the security work you guys are doing is paying off?!
  2. No way can this be true, you work for Microsoft, so how can you be unbiased?
  3. What about Apache 1.3.x?
  4. Does this include SSL?

The first comment makes sense. Since SD3+C has been pushed on campus, we are seeing a lot of positive changes in the attack surface and defect levels of newer product. Thats a good thing. (Go ahead Martha... sue me from jail)

The second comment is typical FUD deflection. Secunia is its own company, and not impacted or have research enforced by Microsoft. If anything, sometimes their reports are very critical of Microsoft... as they should be.

The third comment is interesting. People want to always compare apples to oranges, not giving a fair comparision. They do this at the OS level all the time. Lets compare the latest of both when doing such analysis. But in case thats not a good enough reason for you, you can look at the difference, comparing against Apache 1.3x:


The final comment was about SSL. I was suprised people would want to open this can of worms with all the recent OpenSSL issues. Michael pointed out some interesting stats on that as well. Quoting his view on this:

Microsoft issued a security update, MS04-011 (http://www.microsoft.com/technet/security/Bulletin/MS04-011.mspx) in Feb04 for Windows, which included a bug fix for Private Comms Technology (PCT). PCT was released just after SSL2 to fix a number of defects in the protocol, these were then fixed in SSL3. PCT also support strong crypto for finanical orgs and was enabled by default on all platforms except Windows Server 2003 and Windows XP SP2. So chances are very good if you’re running a new Windows Server 2003 box, you’re not vulnerable because the code path is not exposed by default. So it’s a low pri bug. That said, let’s call it three security bugs related to IIS6."

Now let’s look at Apache2, plus OpenSSL 0.9.x because mod_ssl uses OpenSSL:

Some interesting findings. As an Apache fan I don't like to admit it, but IIS6 has come a long way.

Posted by SilverStr at October 19, 2004 11:54 AM | TrackBack
Comments

I think that saying "IIS has come a long way" is somewhat misleading. The study does nothing to account for current deployment numbers. Apache 2.0.x is widely deployed in production. Comparatively, IIS6 hasn't been out for as long and hasn't been deployed as heavily. I saw a research study, (I wish I could remember where) that pointed out that IIS5 is still far and away the dominant Microsoft webserver in use today. I don't think you will see too many advisories released for IIS6 until it eclipses IIS5 and becomes the target of virus and worm authors.

Posted by: Scott Banwart at October 19, 2004 01:37 PM

Ok, lets use IIS5 as the sample. Take a look at the image at http://secunia.com/graph/?type=adv&prod=39&period=all

As you can see in the same time period comparing against Apache, IIS5 had 7 defects where Apache 2.x had what appears to be 20 defects.

Its a fact; Microsoft's web server is getting better. My only issue is that it is getting so big, and its MUCH to easy to misconfigure it.

Posted by: SilverStr at October 19, 2004 01:59 PM

As has been pointed out on Michael's blog; the numbers don't necessarily speak for themselves.

Firstly, the only people examining the IIS6 code work for Microsoft. This has two side-effects: firstly, bugs don't necessarily get reported publicly, and secondly, if Microsoft doesn't find the bug, it's found by raw exploit.

Secondly, there's the privilege-level issue. Apache almost always runs as a user with permission to do nothing. IIS did, at least, run as LocalSystem.

Thirdly, the figures give no indication of severity. Nor do they give indication of time-taken-to-fix.

I'm not saying IIS6 isn't improving - my gut feeling is that it is, but that it's impossible to draw that conclusion just from those figures.

Posted by: Mo at October 19, 2004 02:45 PM

Those are all good points Mo. I cannot argue any of them.

These figures should be used as a guidance, not as a truth table. However, in the same vein I don't think we can be so quick to brush off IIS. As it continues to get refactored, its getting stronger.

I would like to point out I still use Apache. I still believe for my needs it is the better web server. However, I am taking a double look now at IIS as I see these numbers. Compare them to a few years ago; there was no comparision.

Posted by: SilverStr at October 19, 2004 03:01 PM

It appears that this argument (as any OSS vs. Microsoft debate eventually does) is coming down to not comparing apples to oranges.

Guy Gervais had a good point in that there's many unknown bugs in the MS code bases, but also many known defects that have not been fixed: http://www.guninski.com/browsers.html

I can't really argue whether IIS6 is better then Apache2, but I definetly agree that it's much harder to securly program for and administer IIS5 or IIS6 then Apache.

Posted by: Wim at October 19, 2004 07:33 PM

The issue(s) I have with Microsoft products is that they are tightly integrated. That may well be heaven for developers when it comes to ease of development or time to market, but it sure is hell for security people and, coming full circle, for developers as well: if there is a bug in a tightly integrated piece of software, it's *much* harder to regression test the bugfix. One does not really know at which place in the OS the fixed piece of code may actually break something else.

That's the beauty of any modular system. Anything's b0rken? Replace it without breaking something else.

Posted by: Axel at October 20, 2004 02:00 AM

I have apps on both IIS6 and IIS5 at work. I am a developer but obviously having web apps means I must work closely with the network admins setting up the servers to run my web apps.

They are getting better (MS IIS) but as folks pointed out already they are a configuration nightmare. Forget one thing or configure it wrong and everything else breaks. Then you have to actually pin down wether it's the apps breaking or if IIS is breaking the apps. What runs great on IIS5 often takes days to get set up on IIS6.

So far nothing has moved cleanly. Not that it required any app changes but it just straight wouldn't run until we had everything perfectly configured.

Then you have the late night thoughts of "well I got the apps up and running but did I leave any security holes doing it?".

Where I work it is primarily a WIN-TEL shop so IIS is the defacto web server standard. I would rather work with Apache any day of the week than IIS. This is from a developer standpoint of ease of maintenance and ease of setup.

Security wise I can't say which I'd prefer because we've never been exploited on either one "knock on wood that we continue to keep everything this tightly secured". My two cents on the issue.

Posted by: Andy at October 20, 2004 07:31 AM

Also don't forget (and it's probably been mentioned above) is how bad is the bug for the advisory. IE: is the advisory for a DOS exploit or a 'deface your website' exploit? A quick look at apache's list of 1.3 vulnerabilites at http://www.apacheweek.com/features/security-13 shows a lot of DOS issues mixed in there. Not that there aren't any "real" bugs, but they aren't all "remote users can control your computer" type problems.

A look for a list of II6 vulnerabilties sends me to this page:
http://msmvps.com/bernard/archive/0001/01/01/7882.aspx
which talks about how IIS6 has 60 vulnerabilities, or 48, or 2, or 0, depending on how you count.

Basically I'm saying that whether you're bought off by MS or not, you can manipulate the numbers to show whatever you want ('get the truth' site as a prime example :)

Posted by: Arcterex at October 21, 2004 09:09 AM