Sometime in the past day or so, CVE-2016-10001 was publicly disclosed, and possibly a duplicate. Regardless, CVE-2016-10002 is also now public and legitimate. Tonight, I Tweeted that the presence of those IDs doesn’t mean what many will think it means. I say that based on the past experience, both historical and more recent. Even 17 years later, many people believe that CVE assignments are sequential and that a given ID means that is the number of vulnerabilities aggregated by MITRE that year. That isn’t how it works and it never has.
As of the December 18 dump available from MITRE, there are 10,137 identifiers in the dump. However, 44 of them are REJECTED and 4,760 are in RESERVED status. That means there are 5,333 live CVE identifiers at this time that correspond to vulnerabilities. Since a single CVE ID can include multiple similar vulnerabilities, that number is also misleading. If you take their data and abstract it on a per-vulnerability basis, they cover 8,058 issues as aggregated by Risk Based Security’s VulnDB. So, to be very clear:
CVE has not cataloged 10,000 vulnerabilities in 2016 based on CVE IDs.
Additionally, to be very clear again:
CVE has not cataloged 10,000 vulnerabilities in 2016 based on their actual aggregated vulnerability data.
Meanwhile, VulnDB has currently cataloged 14,485 vulnerabilities, compared to the CVE 8,058 actual number. Hopefully your organization uses more than just CVE data. That means within your security products that scan for vulnerabilities, your tools that collect the data, and ultimately the reporting that guides your security team in making decisions.
All of that said… taking bets if we see Tweets, blogs, or news articles claiming the “10,000 vulns in 2016” notion.
Today, The Register wrote an article on MITRE’s announcement of a new CVE ID scheme, and got many things wrong about the situation. As I began to write out the errata in an email, someone asked that I make it public so they could learn from the response as well.
From The Register:
The pilot platform will implement a new structure for issuance of Common Vulnerabilities and Exposures numbers. MITRE will directly issue the identifiers and bypass its editorial board.
The old system bypassed the Editorial Board. The Board isn’t there to give input on assignments, not for over 10 years. The criticism right now is that the new system was rolled out without consulting the Board, which the board is supposedly there for:
The MITRE Corporation created the CVE Editorial Board, moderates Board discussions, and provides guidance throughout the process to ensure that CVE serves the public interest.
The last ID scheme change was after more than a year of discussion, debate, and ultimately a vote by the board (11/2/2012 Start –> 6/7/2013 End). This ID scheme change was a knee-jerk response to recent criticism (only after it became more public than the Editorial Board list), without any board member input. From The Register article:
The CVE numbers are the numerical tags assigned to legitimate verified bugs that act as a single source of truth for security companies and engineers as they seek to describe and patch problems.
No… CVE IDs are assigned to security issues, period. Alleged, undetermined, possible, or validated. Many IDs are assigned to bogus / illegitimate issues, which are generally discovered after the assignment has been made. There is no rule or expectation that IDs are only assigned to legitimate verified bugs. To wit, search the CVE database for the word “REJECTED”.
More importantly, the notion it is a “single source of truth” is perhaps the worst characterization of CVE one could imagine. Many companies use alternate vulnerability databases that offer more features, better consumption methods, and/or much better coverage. With MITRE falling behind other VDBs by as many as 6,000 vulnerabilities in 2015 alone, it isn’t a single source for anything more than training wheels for your vulnerability program.
The platform will exist alongside the current slower but established CVE system.
At this point, with what has been published by MITRE, this is pure propoganda in line with the platitudes they have been giving the Editorial Board for the last year. The new format does not make things easier for them, in any way. The ‘old ID scheme’ was never the slowdown or choke point in their process. In fact, read their press release and they say they will be the only ones issuing federated IDs, meaning the same problem will happen in the interim. If it doesn’t, and they start issuing faster, it wasn’t the new scheme that fixed it. More important to this part of the story is that it wasn’t just about assignment speed. The last six months have seen MITRE flat out refuse assignments to more and more researchers, citing the vulnerable software isn’t on their list of monitored products. Worse, they actually blame the Editorial Board to a degree, which is a blatant and pathetic attempt at scape-goating. The list they refer to was voted on by the board, yes. But the list given to the board was tragically small and it was about arranging the last chairs on the Titanic so to speak.
He says MITRE is aiming for automated vulnerability identification, description, and processing, and welcomed input from board members and the security community.
This is amusing, disgusting, and absurd. MITRE, including the new person in the mix (Sain) have NOT truly welcomed input from the board. The board has been giving input, non-stop, and MITRE has been ignoring it every single time. Only after repeated mails asking for an update, do they give us the next brief platitude. Sain’s title, “MITRE CVE communications and adoption lead”, is also ironic, given that just last week Sain told the board he would contact them via telephone to discuss the issues facing CVE, and never did. Instead, there was no communication, and MITRE decided to roll out a scheme that was horribly designed without any input from the Board or the industry.
For those who want a better glimpse into just how bad MITRE is handling the CVE project, I encourage you to read the Editorial Board traffic (and hope it is updated, as MITRE still manually runs a script to update that archive).
CVE just announced reaching 30,000 identifiers which is a pretty scary thing. CVE staff have a good eye for catching vulnerabilities from sources away from the mainstream (e.g. bugtraq) and they have the advantage of being a very widely accepted standard for tracking vulnerabilities. As companies and researchers request CVE numbers for disclosures, they get a lot of the information handed to them on a silver platter. Of course, sometimes that platter is full of mud and confusion as vendors don’t always provide clear details to help CVE accurately track and distinguish between multiple vulnerabilities. I’ve also pointed out many times in the past that CVE is a very unique VDB that provides identifiers for vulnerability tracking. They do not provide many fields associated with other VDBs (solution, creditee, etc). As such, they may have a single entry that covers multiple distinct vulnerabilities if they are the same class (XSS, SQLi, RFI), or if there is a lack of details but they know it affects the same product (Oracle). So when we see 30,000 identifiers, we have to realize that the real count of vulnerabilities is significantly higher.
CVE is run by The MITRE Corporation, sponsored / funded by the NCSD (US-CERT) of DHS under government contract. That means our tax dollars fund this database so it should be of particular interest to U.S. taxpayers in the security industry. I know from past discussions with CVE staff and other industry veterans that on any given day, they are more likely to have more work than available staff. That means the rate of vulnerabilities that get published is greater than the resources CVE can maintain to track them. In short, the 30,000 identifiers you see only represents a percentage of the vulnerabilities actually disclosed. We could probably debate what percentage that represents all day long, and I don’t think that is really the point here other than “we know it isn’t all of them”.
Every VDB suffers from the same thing. “Commercial” VDBs like X-Force, BID and Secunia have a full time staff that maintain their databases, like CVE does. Despite having all of these teams (some of them consisting of 10 or more people) maintain VDBs, we still see countless vulnerabilities that are ‘missed’ by all of them. This is not a slight against them in any way; it is a simple manner of resources available and the amount of information out there. Even with a large team sorting disclosed vulnerabilities, some teams spend time validating the findings before adding them to the database (Secunia), which is an incredible benefit for their customers. There is also a long standing parasitic nature to VDBs, with each of them watching the others as best they can, to help ensure they are tracking all the vulnerabilities they can. For example, OSVDB keeps a close eye on Secunia and CVE specifically, and as time permits we look to X-Force, BID, SecurityTracker and others. Each VDB tends to have some researchers that exclusively disclose vulnerabilities directly to the VDB of their choice. So each one I mention above will get word of vulnerabilities that the rest really have no way of knowing about short of watching each other like this. This VDB inbreeding (I will explain the choice of word some other time) is an accepted practice and I have touched on this in the past (CanSecWest 2005).
Due to the inbreeding and OSVDB’s ability to watch other resources, it occasionally frees up our moderators to go looking for more vulnerability information that wasn’t published in the mainstream. This usually involves grueling crawls through vendor knowledge-bases, mind-numbing changelogs, searching CVS type repositories and more. That leads to the point of this lengthy post. In doing this research, we begin to see how many more vulnerabilities are out there in the software we use, that escapes the VDBs most of the time. Only now, after four years and getting an incredible developer to make many aspects of the OSVDB wish-list a reality, do we finally begin to see all of this. As I have whined about for those four years, VDBs need to evolve and move beyond this purely “mainstream reactionary” model. Meaning, we have to stop watching the half dozen usual spots for new vulnerability information, creating our entries, rinsing and repeating. There is a lot more information out there just waiting to be read and added.
In the past few weeks, largely due to the ability to free up time due to the VDB inbreeding mentioned above, we’ve been able to dig into a few products more thoroughly. These examples are not meant to pick on any product / VDB or imply anything other than what is said above. In fact, this type of research is only possible because the other VDBs are doing a good job tracking the mainstream sources, and because some vendors publish full changelogs and don’t try to hide security related fixes. Kudos to all of them.
Example: Search your favorite VDB for ”inspircd”, a popular multi-platform IRC daemon. Compare the results of BID, Secunia, X-Force, SecurityTracker, and http://osvdb.org/ref/blog/inspircd-cve.png. Compare these results to OSVDB after digging into their changelogs. Do these same searches for “xfce” (10 OSVDB, 5 max elsewhere), “safesquid” (6 OSVDB, 1 max elsewhere), “beehive forum” (27 OSVDB, 8 max elsewhere) and “jetty” (25 OSVDB, 12 max elsewhere). Let me emphasize, I did not specifically hand pick these examples to put down any VDB, these are some of the products we’ve investigated in the last few weeks.
The real point here is that no matter what vulnerability disclosure statistic you read, regardless of which VDB it uses (including OSVDB), consider that the real number of vulnerabilities disclosed is likely much higher than any of us know or have documented. As always, if you see vulnerabilities in a vendor KB or changelog, and can’t find it in your favorite VDB, let them know. We all maintain e-mail addresses for submissions and we all strive to be as complete as possible.
Document version: 1.0 Date: October 4, 2006
For the past 5 years, CVE has been tracking the types of errors that lead to publicly reported vulnerabilities, and periodically reporting trends on a limited scale. In support of the Common Weakness Enumeration (CWE) project, and as a result of the interest in this work as mentioned during the “Year of the web application: Hack & Data from the Front lines” panel at the 5th Annual Cyber Security Executive Summit in New York City on September 13, 2006, we have published a more extensive analysis. An updated version will be released once 2006 is complete.
The primary goal of this study is to better understand research trends using publicly reported vulnerabilities. It should be noted that the data is obtained from an uncontrolled population, i.e., decentralized public reports from a research community with diverse goals and interests, with an equally diverse set of vendors and developers. More specialized, exhaustive, and repeatable methods could be devised to evaluate software security. But until such methods reach maturity and widespread acceptance, the overall state of software security can be viewed through the lens of public reports.
Steve Christey of CVE has posted to several lists asking What is the state of vulnerability research? Before you dismiss the question, give it serious thought for a few minutes. Have any ideas, opinions or concerns about where vuln research is heading? Where it should be? Drop him a line and let him know.
One person challenged him stating that if MITRE were the experts they proclaim, he wouldn’t have to ask. After a few years of being heavily involved with vulnerability databases and monitoring such research, I of course had to reply.