Elinor Mills wrote an article titled Firefox, Adobe top buggiest-software list. In it, she quotes Qualys as providing vulnerability statistics for Mozilla, Adobe and others. Qualys states:
The number of vulnerabilities in Adobe programs rose from 14 last year to 45 this year, while those in Microsoft software dropped from 44 to 41, according to Qualys. Internet Explorer, Windows Media Player and Microsoft Office together had 30 vulnerabilities.
This caught my attention immediately, as I know I have mangled more than 45 Adobe entries this year.
First, the “number of vulnerabilities” game will always have wiggle room, which has been discussed before. A big factor for statistic discrepancy when using public databases is the level of abstraction. CVE tends to bunch up vulnerabilities in a single CVE, where OSVDB tends to break them out. Over the past year, X-Force and BID have started abstracting more and more as well.
Either way, Qualys cited their source, NVD, which is entirely based on CVE. How they got 45 vulns in “Adobe programs” baffles me. My count says 97 Adobe vulns, 95 of them have CVEs assigned to them (covered by a total of 93 CVEs). OSVDB abstracted the entries like CVE did for the most part, but split out CVE-2009-1872 as distinct XSS vulnerabilities. OSVDB also has two entries that do not have CVE, 55820 and 56281.
Where did Qualys get 45 if they are using the same CVE data set OSVDB does? This discrepancy has nothing to do with abstraction, so something else appears to be going on. Doing a few more searches, I believe I figured it out. Searching OSVDB for “Adobe Reader” in 2009 yields 44 entries, one off from their cited 45. That could be easily explained as OSVDB also has 9 “Adobe Multiple Products” entries that could cover Reader as well. This may in turn be a breakdown where Qualys or Mills did not specify “Adobe Software” (cumulative, all software they release) versus “Adobe Reader” or some other specific software they release.
Qualys tallied 102 vulnerabilities that were found in Firefox this year, up from 90 last year.
What is certainly a discrepancy due to abstraction, OSVDB has 74 vulnerabilities specific to Mozilla Firefox (two without CVE), 11 for “Mozilla Multiple Browsers” (Firefox, Seamonkey, etc) and 81 for “Mozilla Multiple Products” (Firefox, Thunderbird, etc). While my numbers are somewhat anecdotal, because I cannot remember every single entry, I can say that most of the ‘multiple’ vulnerabilities include Firefox. That means OSVDB tracked as many as, but possibly less than, 166 vulnerabilities in Firefox.
Microsoft software dropped from 44 to 41, according to Qualys. Internet Explorer, Windows Media Player and Microsoft Office together had 30 vulnerabilities.
According to my searches on OSVDB, we get the following numbers:
- 234 vulnerabilities in Microsoft, only 4 without CVE
- 50 vulnerabilities in MSIE, all with CVE
- 4 vulnerabilities in Windows Media Player, 1 without CVE
- 52 vulnerabilities in Office, all with CVE. (based on “Office” being Excel, Powerpoint, Word and Outlook.
- 92 vulnerabilities in Windows, only 2 without CVE
When dealing with vulnerability numbers and statistics, like anything else, it’s all about qualifying your numbers. Saying “Adobe Software” is different than “Adobe Acrobat” or “Adobe Reader” as the software installation base is drastically different. Given the different levels of abstraction in VDBs, it is also equally important to qualify what “a vulnerability” (singular) is. Where CVE/NVD will group several vulnerabilities in one identifier, other databases may abstract and assign unique identifiers to each distinct vulnerability.
Qualys, since you provided the stats to CNet, could you clarify?
I always mean to post changes more frequently, but apathy and other tasks seem to win the day. Here is a brief list of OSVDB change highlights over the past few months.
- The database currently covers 59,833 vulnerabilities, spanning 26,179 products from 4,735 researchers, over 44 years.
- Sequoia, ES&S and InkaVote e-voting machine audit documents integrated (all text search “electronic voting machine”)
- Happy 10th birthday CVE! We are now “fully” mapped for all years
- Integrated the historic Phage mail list content (http://securitydigest.org/phage/)
- Our Snort X-ref import script was borked (since Sep). Fixed, added almost 500 recent Snort IDs to references
- Apache bug system scoured, over 150 Apache vulns added from the last eight years (http://osvdb.org/ref/blog/apache-scouring.txt)
- Metasploit static references supported (http://blog.osvdb.org/2009/10/23/metasploit-reference-support-added-more)
- Exploit-DB references supported (http://www.exploit-db.com/)
- VuPEN references supported (http://www.vupen.com/)
- Many vulnerability and solution templates overhauled
- Search engine rebuilds are considerably faster, will auto-tweet when rebuilding (as it may affect search results)
- Reference search for full URL works
- Title search for multiple words fixed (was temporarily matching on some but not all words)
- New search filters and custom exports (http://blog.osvdb.org/2009/11/09/search-filters-custom-exports)
- Inverse search filtering enabled (http://blog.osvdb.org/2009/10/30/not-it)
- Search by CVSS scores (http://blog.osvdb.org/2009/10/28/search-enhance-by-cvss-score-or-attribute)
- Any search can be turned into a ‘Watch List’. Left nav menu has this option, new results are mailed to you as entered in the system
- New menu system (top and left nav)
- Twitter feed more actively used for project updates
- Twitter feed displays on front page
- ‘About’ page is updated, expect more static pages to be updated to better reflect project status soon
- CVSSv2 scoring support added, including:
- CVSS scoring history (historically track NVD, OSVDB and other sources)
- Anyone can submit scores for entries without CVE/NVD (over 13,000)
- Updating CVSS scores for entries without are worth .25 points for now, to encourage mangling
- Moderation system in place for submitted CVSS scores
- Creditee system overhaul (http://blog.osvdb.org/2009/11/21/creditee-system-overhauled)
- “Vulnerabilities in OSVDB disclosed by type by quarter” graphs added to front page
- More fixes to continue support for IE6. Don’t expect this to last!