I’m big on Vulnerability Database (VDB) evolution. I tend to harp on them for not adding features, not making the data more accessible and generally doing the exact same thing they did ten years ago. While the target of my ire is typically functionality or usability, today it is about a little more.
Last night I wanted to check for details on a CVE entry that was rather vague and had a single reference to BID. This is fairly common in the VDB world as one database will add an entry and not provide a link to the source of the data (Secunia and BID primarily). As luck would have it, BID was down. Almost twelve hours later and their VDB is still down. What annoys me is that while they aren’t delivering vulnerability information, they sure are delivering advertisements. Why can’t VDBs get the same dedication and resources that ad farms get?
Next, I wanted to find out if the other VDBs created an entry for the latest OpenBSD flap yet, so I went to X-force which is a pretty reliable database. Much to my dismay, it appears that the ‘advanced’ search is now gone. While it wasn’t extremely powerful, it let you do some basic sorting that was immensely helpful in finding what you need. I have mail out to them asking for confirmation that it is indeed gone versus a web geek error. I certainly hope it is the latter…
Update: Over 24 hours later, the BID database is finally available again. ISS has not replied to at least two mails from VDB managers asking about the missing advanced search feature.
CVE just announced reaching 30,000 identifiers which is a pretty scary thing. CVE staff have a good eye for catching vulnerabilities from sources away from the mainstream (e.g. bugtraq) and they have the advantage of being a very widely accepted standard for tracking vulnerabilities. As companies and researchers request CVE numbers for disclosures, they get a lot of the information handed to them on a silver platter. Of course, sometimes that platter is full of mud and confusion as vendors don’t always provide clear details to help CVE accurately track and distinguish between multiple vulnerabilities. I’ve also pointed out many times in the past that CVE is a very unique VDB that provides identifiers for vulnerability tracking. They do not provide many fields associated with other VDBs (solution, creditee, etc). As such, they may have a single entry that covers multiple distinct vulnerabilities if they are the same class (XSS, SQLi, RFI), or if there is a lack of details but they know it affects the same product (Oracle). So when we see 30,000 identifiers, we have to realize that the real count of vulnerabilities is significantly higher.
CVE is run by The MITRE Corporation, sponsored / funded by the NCSD (US-CERT) of DHS under government contract. That means our tax dollars fund this database so it should be of particular interest to U.S. taxpayers in the security industry. I know from past discussions with CVE staff and other industry veterans that on any given day, they are more likely to have more work than available staff. That means the rate of vulnerabilities that get published is greater than the resources CVE can maintain to track them. In short, the 30,000 identifiers you see only represents a percentage of the vulnerabilities actually disclosed. We could probably debate what percentage that represents all day long, and I don’t think that is really the point here other than “we know it isn’t all of them”.
Every VDB suffers from the same thing. “Commercial” VDBs like X-Force, BID and Secunia have a full time staff that maintain their databases, like CVE does. Despite having all of these teams (some of them consisting of 10 or more people) maintain VDBs, we still see countless vulnerabilities that are ‘missed’ by all of them. This is not a slight against them in any way; it is a simple manner of resources available and the amount of information out there. Even with a large team sorting disclosed vulnerabilities, some teams spend time validating the findings before adding them to the database (Secunia), which is an incredible benefit for their customers. There is also a long standing parasitic nature to VDBs, with each of them watching the others as best they can, to help ensure they are tracking all the vulnerabilities they can. For example, OSVDB keeps a close eye on Secunia and CVE specifically, and as time permits we look to X-Force, BID, SecurityTracker and others. Each VDB tends to have some researchers that exclusively disclose vulnerabilities directly to the VDB of their choice. So each one I mention above will get word of vulnerabilities that the rest really have no way of knowing about short of watching each other like this. This VDB inbreeding (I will explain the choice of word some other time) is an accepted practice and I have touched on this in the past (CanSecWest 2005).
Due to the inbreeding and OSVDB’s ability to watch other resources, it occasionally frees up our moderators to go looking for more vulnerability information that wasn’t published in the mainstream. This usually involves grueling crawls through vendor knowledge-bases, mind-numbing changelogs, searching CVS type repositories and more. That leads to the point of this lengthy post. In doing this research, we begin to see how many more vulnerabilities are out there in the software we use, that escapes the VDBs most of the time. Only now, after four years and getting an incredible developer to make many aspects of the OSVDB wish-list a reality, do we finally begin to see all of this. As I have whined about for those four years, VDBs need to evolve and move beyond this purely “mainstream reactionary” model. Meaning, we have to stop watching the half dozen usual spots for new vulnerability information, creating our entries, rinsing and repeating. There is a lot more information out there just waiting to be read and added.
In the past few weeks, largely due to the ability to free up time due to the VDB inbreeding mentioned above, we’ve been able to dig into a few products more thoroughly. These examples are not meant to pick on any product / VDB or imply anything other than what is said above. In fact, this type of research is only possible because the other VDBs are doing a good job tracking the mainstream sources, and because some vendors publish full changelogs and don’t try to hide security related fixes. Kudos to all of them.
Example: Search your favorite VDB for ”inspircd”, a popular multi-platform IRC daemon. Compare the results of BID, Secunia, X-Force, SecurityTracker, and http://osvdb.org/ref/blog/inspircd-cve.png. Compare these results to OSVDB after digging into their changelogs. Do these same searches for “xfce” (10 OSVDB, 5 max elsewhere), “safesquid” (6 OSVDB, 1 max elsewhere), “beehive forum” (27 OSVDB, 8 max elsewhere) and “jetty” (25 OSVDB, 12 max elsewhere). Let me emphasize, I did not specifically hand pick these examples to put down any VDB, these are some of the products we’ve investigated in the last few weeks.
The real point here is that no matter what vulnerability disclosure statistic you read, regardless of which VDB it uses (including OSVDB), consider that the real number of vulnerabilities disclosed is likely much higher than any of us know or have documented. As always, if you see vulnerabilities in a vendor KB or changelog, and can’t find it in your favorite VDB, let them know. We all maintain e-mail addresses for submissions and we all strive to be as complete as possible.
Jason Bergen posted to Full-Disclosure trying to sell a “Security Vulnerability Database Company“. From that mail:
The company maintains a database of all security vulnerabilities, and the database is updated on a daily basis. The company maybe of interest to organisations who are currently licensing a vulnerability database. In addition the company has developed some software applications built upon the vulnerability database.
This is interesting on many levels, especially the approach in selling it. Why post to that mail list and not others? When asked for more details, Mr Bergen tells you “In order to provide further information a signed NDA would be required.” You must sign a non-disclosure agreement just to find out the name of the company being sold. He also makes the following claim:
The database contains all vulnerabilities since 1988. Each entry has Bugtraq, CVE, and Nessus ids. It has developed its own vulnerability alerting system, but recently changed focus to providing OEM database licensing.
Sadly, he is not the first to make this claim. Throughout the years, many people have referred to CVE as having “all vulnerabilities since 1988” which simply is not the case. If you ask Steve Christey or anyone involved with CVE, they will be the first to tell you that isn’t the case. So why do people think that? CERT started releasing advisories in 1988, but only released them for serious/critical vulnerabilities. Between 1988 and 1999 (CVE inception), many vulnerabilities were never added or given a formal advisory for. In short, claims that their database has “all vulnerabilities since 1988” is extremely suspect. Had it been any year other than 1988, perhaps they took the time to go back and add them making the claim true. His wording also begs the question, what if a vulnerability doesn’t have a BID, CVE or Nessus ID to match? As much as databases try to maintain a perfect cross reference mapping, it just doesn’t happen all the time.
From: Steven M. Christey (coley[at]mitre.org)
Date: Fri Jul 15 2005 – 13:35:52 CDT
Vulnerability databases and notification services have to pore through approximately 100 new public vulnerability reports a week. Correction: that’s HUNDREDS of reports, from diverse and often unproven sources, for about 100 unique vulnerabilities per week.
A LARGE number of vendors and maintainers either:
Document Detailing “CVE Content Decisions” Now Available
June 15, 2006 — A new document entitled “CVE Abstraction Content Decisions: Rationale and Application” detailing CVE content decisions (CDs) has been posted on the CVE Web site. CVE CDs are the guidelines used to ensure that CVE names are created in a consistent fashion, independent of who is doing the creation.