Hopefully a really quick blog, but a section of a news article titled “Hackers are having a field day with stolen credentials” by Amol Sarwate, Qualys’ Director of Vulnerability Labs, published in SC Magazine caught my attention. The section:
Let’s X-ray the attack methods
Typically, hackers “fingerprint” websites’ underlying software, such as their blog content management system or discussion forum application, and exploit either known vulnerabilities the website owner left unpatched or zero-day flaws.
In one case, an attacker used misplaced install files to gain admin privileges. In another case, hackers stole one moderator’s credentials and used the account to post a malicious message in the forum. After viewing the message, the forum’s administrator had his account compromised, leading to a massive breach. Notable vulnerabilities exploited in recent years include CVE-2016-6483, CVE-2016-6195, CVE-2016-6635, CVE-2015-1431, CVE-2015-7808, CVE-2014-9574 and CVE-2013-6129.
Specifically, that list of CVE identifiers. First, a random list of CVE IDs and they don’t even link to the entries on the CVE or NVD site. It’s not like anyone will instantly recognize those and equate them to specific vulnerabilities, and the odds of someone cutting and pasting them into Google are slim. Second, in the context mentioned, talking about exploiting web sites, then mentioning stealing credentials and account compromise, the list is peculiar. Looking them up in VulnDB:
142673 2016-08-01 2016-6483 vBulletin /link/getlinkdata Server-side Request Forgery (SSRF)
141687 2016-07-11 2016-6195 vBulletin forumrunner/includes/moderation.php Multiple Parameter SQL Injection
137861 2016-03-30 2016-6635 WordPress wp-admin/includes/ajax-actions.php Script Compression Option CSRF
129847 2015-11-02 2015-7808 vBulletin /vbforum/ajax/api/hook/decodeArguments arguments Parameter Remote Code Execution
117888 2015-01-26 2015-1431 phpBB includes/startup.php Trailing Path Handling CSS Injection
116744 2014-12-31 2014-9574 FluxBB /install.php require() Function install_lang Parameter Path Traversal Local File Inclusion
98370 2013-10-09 2013-6129 vBulletin install/upgrade.php Configuration Mechanism Admin Account Creation
A few observations:
- While SSRF can be an underlooked vulnerability, it often leads to remote information disclosure about other hosts, not specifically the web site in Sarwate’s context.
- SQL Injection we’d certainly expect to see and can have devastating consequences.
- CSRF is another as it can be used to conduct privileged operations via phishing attacks.
- Any Remote Code Execution issue is obviously a big one, and on widely deployed software like vBulletin it is expected on such a list.
- CSS injection is an odd one too, as it is frequently considered less severe than XSS which doesn’t appear on the list at all, despite Sarwate describing an XSS in the example above.
- While LFI can be a serious vuln, curious to see it here instead of a RFI which gives the attacker far more control and more reliably results in a compromise.
- 2013-6129 was discovered in the wild and being actively exploited, so no surprise to see it in such a list. Curious it is from 2013 and there are no more recent examples he thought to use.
- Seven issues, with varying degrees of severity, some that may not pose a serious risk to a web site, while leaving out XSS completely, an RCE in Joomla! that was being exploited in the wild, various WordPress plugins that allowed for RCE, etc.
Anyway, just found this list odd and figured it was worth the mention.
[Sent to Ashley directly via email. Posting for the rest of the world as yet another example of how vulnerability statistics are typically done poorly. In this case, a company that does not aggregate vulnerabilities themselves, and has no particular expertise in vulnerability metrics weighs in on 2013 “statistics”. They obviously did not attend Steve Christey and my talk at BlackHat last year titled “Buying Into the Bias: Why Vulnerability Statistics Suck“. If we do this talk again, we have a fresh example to use courtesy of Skybox.]
[Update: SkyboxSecurity has quickly written a second blog in response to this one, clarifying a lot of their methodology. No word from Carman or SC Magazine. Not surprised; they have a dismal history as far as printing corrections, retractions, or even addressing criticism.]
In your recent article “Microsoft leads vendors with most critical vulnerabilities“, you cite research that is factually incorrect, and I fully expect a retraction to be printed. In fact, the list of errata in this article is considerably longer than the article itself. Some of this may seem to be semantics to you, but I assure you that in our industry they are anything but. Read down, where I show you how their research is *entirely wrong* and Microsoft is not ‘number one’ here.
1. If Skybox is only comparing vendors based on their database, as maps to CVE identifiers, then their database for this purpose is nothing but a copy of CVE. It is important to note this because aggregating vulnerability information is considerably more demanding than aggregating a few databases that do that work for you.
2. You say “More than half of the company’s 414 vulnerabilities were critical.” First, you do not disclaim that this number is limited to 2013 until your last paragraph. Second, Microsoft had 490 disclosed vulnerabilities in 2013 according to OSVDB.org, apparently not one of the “20” sources Skybox checked. And we don’t claim to have all of the disclosed vulnerabilities.
3. You cite “critical vulnerability” and refer to Microsoft’s definition of that as “one that allows code execution without user interaction.” Yet Skybox did not define ‘critical’. This is amateur hour in the world of vulnerabilities. For example, if Microsoft’s definition were taken at face value, then code execution in a sandbox would still qualify, while being considerably less severe than without. If you go for what I believe is the ‘spirit’ of the research, then you are talking about vulnerabilities with a CVSS score of 10.0 (network, no user interaction, no authentication, full code execution to impact confidentiality / integrity / availability completely), then Microsoft had 10 vulnerabilities. Yes, only 10. If you add the ‘user interaction’ component, giving it a CVSS score of 9.3, they had 176. That is closer to the ‘216’ Skybox is claiming. So again, how can you cite their research when they don’t define what ‘critical’ is exactly? As we frequently see, companies like to throw around vulnerability statistics but give no way to reproduce their findings.
4. You say, “The lab’s findings weren’t particularly surprising, considering the vendors’ market shares. Microsoft, for instance, is the largest company and its products are the most widely used.” This is completely subjective and arbitrary. While Microsoft captures the desktop OS market share, they do not capture the browser share for example. Further, like all of the vendors in this study, they use third-party code from other people. I point this line out because when you consider that another vendor/software is really ‘number one’, it makes this line seem to
be the basis of an anecdotal fallacy.
5. You finish by largely parroting Skybox, “Skybox analyzed more than 20 sources of data to determine the number of vulnerabilities that occurred in 2013. The lab found that about 700 critical vulnerabilities occurred in 2013, and more than 500 of them were from four vendors.” We’ve covered the ‘critical’ fallacy already, as they never define what that means. I mentioned the “CVE” angle above. Now, I question why you didn’t challenge them further on this. As a security writer, the notion that “20” sources has any meaning in that context should be suspect. Did they simply look to 20 other vulnerability databases (that do all the initial data aggregation) and then aggregate them? Did they look at 20 unique sources of vulnerability information themselves (e.g. the MS / Adobe / Oracle advisory pages)? This matters greatly. Why? OSVDB.org monitors over 1,500 sources for vulnerability information. Monitoring CVE, BID, Secunia, and X-Force (other large vulnerability databases) is considered to be 4 of those sources. So what does 20 mean exactly? To me, it means they are amateurs at best.
6. Jumping to the Skybox blog, “Oracle had the highest total number of vulnerabilities at 568, but only 18 percent of their total vulnerabilities were deemed critical.” This is nothing short of a big red warning flag to anyone familiar with vulnerabilities. This line alone should have made you steer clear from their ‘research’ and demanded you challenge them. It is well known that Oracle does not follow the CVSS standards when scoring a majority of their vulnerabilities. It has been shown time and time again that what they scored is not grounded in reality, when compared to the
researcher report that is eventually released. Every aspect of a CVSS score is frequently botched. Microsoft and Adobe do not have that reputation; they are known for generally providing accurate scoring. Since that scoring is the quickest way to determine criticality, it is important to note here.
7. Now for what you are likely waiting for. If not Microsoft, who? Before I answer that, let me qualify my statements since no one else at this table did. Based on vulnerabilities initially disclosed in 2013, that have a CVSS score of 10.0 (meaning full remote code execution without user interaction), we get this:
Two vendors place higher than Microsoft based on this. Now, if we consider “context-dependent code execution”, meaning that user interaction is required but it leads to full code execution (e.g. click this malicious PDF/DOC/GIF and we base that on a 9.3 CVSS score (CVSS2#AV:N/AC:M/Au:N/C:C/I:C/A:C”)) or full remote code execution (CVSS2#AV:N/AC:L/Au:N/C:C/I:C/A:C) we get the following:
I know, Microsoft is back on top. But wait…
OSF / OSVDB.org
Open Security Foundation Wins the SC Magazine 2009 Editor’s Choice Award
Festivities in San Francisco wrapped up last night, and OSF was presented with SC Magazine’s 2009 Editor’s Choice Award. Thanks to everyone who has supported OSF in the past and present, and we definitely hope you’ll continue to support us in the future!