Mac vs Windows - More "Statistics"
Posted by jericho
Yet another article comparing Mac vs Windows, and using statistics to back it up. Since this is getting to be a common occurance, I won’t go into the usual lecture about statistics, how they can easily be manipulated to back any argument (including how VAX/VMS is the most in/secure OS in the world!), how you must fully qualify the data you used to generate your statistics, and all the other tricks that make statistics the best tool to create a convincing argument (lie?). I’m not saying this because I think Mac or Windows is more or less secure. I’m saying this because I don’t feel the following article is accurate or well written. Even the readers who commented bring up some very valid points and questions for the author. Add to that it seems that the author (George Ou) is somewhat outspoken and a fan of Microsoft, his credibility and bias toward rivals comes into question. I’d love for Secunia to officially respond to this article, since he uses their database and rating system to generate his stats.
George Ou’s relevant conclusions: Between Feb 04 and Feb 06, Mac OS X had 5 “extremely critical” (1 unpatched) vulnerabilities and MS Windows had 2 “extremely critical” (0 unpatched) vulnerabilities. Mac OS X had 173 high and 59 moderate vulns, while MS Windows had 49 high and 41 moderate vulns. Ou goes to conclude ”The data is clear, and Apple has a lot more vulnerabilities of every kind ranging from moderately critical to extremely critical. “
Vulnerability statistics for Mac and Windows http://blogs.zdnet.com/Ou/?p=165
One of many good comments challening the piece: http://www.zdnet.com/[..]messageID=356498&start=-1
Past criticism of Ou’s work, and signs he may be biased toward Microsoft: http://www.google.com/search?hl=en&q=George+Ou
Seems his bias toward Microsoft is not only in infosec, but also in other areas. This brief Linux vs Windows piece on performance from last December is a good read:
http://blogs.zdnet.com/Ou/?p=140
Here is a reader’s reply to his piece:
“On my machine at home Linux boots longer than Windows would. I could really use those seconds to scratch my back or something. So, why did you choose the boot time as a relevant parameter? Because you had to choose something which would show an advantage for Windows, as required by whoever pays you for FUD, and you couldn’t find anything that really matters or you are clueless enough to not know what else to look for. Besides that, how many daemons were started during the boot time on Linux? A lot of stuff is set up to be started by default on Linux, which simply does not exist on Windows. Did you starts an FTP, HTTP, SSH, this and that? Did you forget to consider this on purpose or are you again clueless enough to not know what a daemon is? Of course, if you only use 96 MB of memory, a system that starts a lot of these daemons will get hit badly in terms of performance. I admit that Windows boot faster, but so what? After the boot, what happens then?
Well, after the boot, you chose to run OpenOffice, just because that application is notoriously slow. If OpenOffice is slow, that means that OpenOffice is slow, not that Linux is slow. However, even if we are comparing the performance of OpenOffice and MS Office, which we shouldn’t be, according to what your intension was, once they both start, it is the same. Why didn’t you compare, for example, KOffice, which is very fast, to MS Office, if you want to compare the performance of two applications, in a separate article. I thought that you wanted to compare two operating systems.”
I think the reader’s reply does a good job pointing out that Ou provides statistics, while leaving out important, relevant information.
This Mac vs Windows case may be just another example of that.
http://blogs.zdnet.com/Murphy/?p=542
Better Mac OS X Security Numbers Posted by Paul Murphy @ 3:37 am
Some things just aren’t credible on their face, so when George Ou mined Secunia’s security advisories for vulnerability data to prove that Mac OS X is less secure than Windows/XP, I had an immediate problem. According to his research Secunia’s security advisories since January 2004 cover about 238 serious Mac OS X vulnerabilities and only 95 Windows/XP ones, and a 2.5:1 ratio favoring Windows didn’t seem reasonable.
[..]
Unfortunately that’s not the worst problem with the analysis. The biggest problem is that he invites the reader to draw a wholly erroneous conclusion from his numbers: that Mac OS is less secure than Windows/XP.
[..]
A few more related pieces:
http://blogs.technet.com/msrc/archive/2005/10/17/412636.aspx Notes from the Security Road from Mike Nash
http://www.emergentchaos.com/archives/2005/10/counting_in_com.html Counting In Computer Security (Posted by adam)
While working on the latest Apple vulns, it occured to me that Ou appears to have made a fairly serious mistake when compiling these stats because of the methodology he chose to use. Secunia 19064 covers 15 “Apple” vulnerabilities, and is flagged “Extremely Critical”. As far as I can tell, Ou now counts this as 15 extremely critical Apple vulnerabilities. If so, this seriously skews his statistics, partially due to the way Secunia breaks these out. They tend to make one entry for each apple update, but will break out each Windows entry per Microsoft advisory. This gives the Windows vulnerabilities a more refined risk rating, since each entry can be labeled seperately.
Read the latest Apple advisory entry on Secunia, and it is clear that not all of the issues are ‘extremely critical’. This is something he should have taken into account for, or at the very least, disclaimed more thoroughly in the write up. At the least, this is careless on his part. At worst, this is a fine example of biased statistic manipulation.