Lies, Damn Lies, and Statistics

If you recall my posting last week, I talked about Bill Gates' email revisiting Microsoft's ongoing security initiatives. Near the end I pointed out some of the stats that Bill used to show the significant reduction in serious security related vulnerabilities of Windows Server 2003 in the first 320 days compared to the same time period for Windows 2000.

Well according to an article at vnunet.com Joe Wilcox, an analyst with Jupiter Research, commented that Gates "must have a different way of counting", and refutes Bill's stats. Suprise suprise.

Wilcox said he counted 15 security alerts for Windows Server 2003 since the product shipped in April 2003, which rose to more than 20 when products integrated into Windows, such as Internet Explorer Server 2003, were taken into consideration.

"I figure where there is one counting disagreement, there might be another. So I went back and counted up those Windows 2000 Server security alerts. I came up with 28 during the same span of time I got 15 for Windows Server 2003. Windows 2000 Server reached 15 alerts seven months after launch," he said.

Something else I found interesting was that he added that an upcoming Jupiter Research report on Microsoft security reveals that a mere 36 per cent of IT managers from businesses with revenue of $50m or more feel that Microsoft product security has improved.

So... what have we learned here? 82% of stats cannot be trusted. But don't quote me on that. I made it up. :)