On September 8, 2016, Jason Levy of WhiteSource Software published a blog titled “Open Source Vulnerability Database”. Almost two years later it came across my radar and I asked via Twitter if WhiteSource was interested in getting feedback on the blog, since it contained errata. They never replied and it fell off my radar. Last week, someone asked me about WhiteSource’s claims about maintaining the largest vulnerability database. While I have not seen it, I expressed doubt and dug into my notes. Until then, I had forgotten that I created notes to write a blog listing the errata. Since their name came up again, I figured it is worth revisiting this, especially since they never replied to me.
First, I was a long-term contributor to OSVDB, an officer in OSF (the 501(c)(3) that managed the project), a founder at Risk Based Security, and one of the maintainers of VulnDB since day one. Don’t worry, all of that is explained in more detail below. Here are quotes from their blog and my thoughts on the errata:
The open source community has been living up to this statement recently, with the accelerated rate of discoveries of open source vulnerabilities reported by such sources as the NVD open source vulnerability database.
CVE/NVD have dismal coverage of open source libraries in the big picture. Their coverage is primarily through a couple dozen projects that seek out CVE IDs, researchers that request CVE IDs, and the OSS-Sec mail list. To date, there are still many developers, projects, and even security researchers that don’t know what CVE is.
The NVD is by far the main source for researching vulnerabilities.
This sentence does not make sense. As you note below, NVD does not research vulnerabilities; they don’t even aggregate vulnerabilities themselves. While many consume NVD’s data over CVE, due to the format they make the data available, more “analysis” is done via CVEDetails.com than NVD in my experience.
Their analysis contains information like how the vulnerability operates, its impact rating, CVSS score, and links to any available patches/fixes.
The main purpose of NVD, aside from a burden on taxpayer’s dollars, is to provide CPE and CVSS scores for vulnerabilities via CVE. They rarely, if ever, do additional analysis of the vulnerability unless required to determine a CVSS score. If they do, they do not “show their work” so to speak, instead using the same CVE description. As best I am aware, they don’t dig up additional references either, so the only solution links provided are via CVE. That said, I haven’t audited NVD in several years to see if that still holds true, but I would be surprised if anything has changed.
The OSVDB (open source vulnerability database) was launched in 2004 by Jake Kouhns, the founder and current CISO of Risk Based Security – the company which now operates OSVDB’s commercial version, the VulnDB.
Jake Kouns, one of the founders of Risk Based Security (RBS) did not launch OSVDB. And that is the Open Sourced Vulnerability Database. OSVDB was created and launched by H.D. Moore. More history is available via Wikipedia. While OSVDB was a basis for the historical data in VulnDB, Risk Based Security funded OSVDB entirely for over two years and it was their data that was shared publicly via OSVDB before the project shut down. When OSVDB shut down, RBS continued to maintain VulnDB which contains over eight years of vulnerability information not found in OSVDB. With considerable resources and a team maintaining VulnDB, it has become something OSVDB wished it could, but never came close.
This commercial version continues to be maintained by Risk Based Security, but without the strong community that maintained the OSVDB.
There was no “strong community” that maintained OSVDB. For years, it limped along on the backs of a handful of volunteers, with very little support from the community; financial or labor. At times, there were only two of us maintaining the project while most volunteers signed up, worked on the database for 20 minutes, and never returned to help again.
The OSVDB/VulnDB is perceived by many in the community as redundant, for the majority of vulnerabilities are also reported in the NVD.
This is WhiteSource’s ignorance or bias at play. OSVDB was known for having tens of thousands of vulnerabilities not found in CVE/NVD. One of VulnDBs strong points is that it contains almost 69,000 vulnerabilities not found in CVE/NVD. Since WhiteSource claims to have a comprehensive database, you should have disclaimed this while spreading false information about VulnDB. Since VulnDB is not public, the notion that is perceived by “many” in such a light is just your marketing. Anyone reading VulnDBs quarterly or year-end reports can see the incredible gap between them and CVE/NVD. Also curious is that the link off “monitoring of multiple vulnerability databases including the CVE/NVD” just links to your blog on the topic. Other than CVE/NVD, which vulnerability databases do you monitor exactly?
I’d also happily sign a non-disclosure agreement to do a quick audit of your database, to verify or refute your claim that you have the “the largest coverage of vulnerability listings“. I’d only ask that in return, I can publish a simple ‘yes’ or ‘no’ answer to that claim.
Security advisories are usually the first place that security professionals and developers look when they have security issues within a specific scope.
If this is WhiteSource’s methodology for maintaining a vulnerability database, it does not bode well for your customers. Formal security advisories are quickly becoming, or have already become, a minority in the disclosure game. OSVDB and VulnDB didn’t stay far ahead of the curve using this antiquated mindset. We knew to look elsewhere all along.
WhiteSource tracks almost a dozen security advisories (like RubyOnRails, RubySec and Node Security), to ensure our customers are alerted on new vulnerabilities the minute they’re reported.
A whole dozen? The VulnDB team currently monitors 3,100 sources every week. Some are monitored ~ hourly, some a few times a day, some once a week depending on various criteria such as publication frequency. If WhiteSource is only looking at those sources, and that type of source, it really doesn’t bode well for your claims of the largest vulnerability listing.
The majority of vulnerabilities are published with a remediation suggestion, like a new patch, new version, system configuration suggestion etc.
While technically true, per VulnDB data, it is better to qualify that. For example, in Q1 2019, 60.8% of disclosed vulnerabilities they aggregated had a documented solution. But, a couple caveats. First, that was down 13.5% over Q1 2018! Second, not all solutions are created equal. Just because there is a patch committed against the ‘master’ branch doesn’t mean an organization can actually use it. Their policy or available resources may not allow for one-off code patching like that.
And here at WhiteSource, this is exactly what we do.
Unfortunately, based on the way you describe what you do, you are not much better than CVE/NVD I’m afraid.