Tag Archives: McAfee

The Scraping Problem and Ethics

[2014-05-09 Update: We'd like to thank both McAfee and S21sec for promptly reaching out to work with us and to inform us that they are both investigating the incident, and taking steps to ensure that future access and data use complies with our license.]

Every day we get requests for an account on OSVDB, and every day we have to turn more and more people away. In many cases the intended use is clearly commercial, so we tell them they can license our data via our commercial partner Risk Based Security. While we were a fully open project for many years, the volunteer model we wanted didn’t work out. People wanted our data, but largely did not want to give their time or resources. A few years back we restricted exports and limited the API due to ongoing abuse from a variety of organizations. Our current model is designed to be free for individual, non-commercial use. Anything else requires a license and paying for the access and data usage. This is the only way we can keep the project going and continue to provide superior vulnerability intelligence.

As more and more organizations rely on automated scraping of our data in violation of our license, it has forced us to restrict some of the information we provide. As the systematic abuse rises, one of our only options is to further restrict the information while trying to find a balance of helping the end user, but crippling commercial (ab)use. We spend about half an hour a week looking at our logs to identify abusive behavior and block them from accessing the database to help curb those using our data without a proper license. In most cases we simply identify and block them, and move on. In other cases, it is a stark reminder of just how far security companies will go to to take our information. Today brought us two different cases which illustrate what we’re facing, and why their unethical actions ultimately hurt the community as we further restrict access to our information.

This is not new in the VDB world. Secunia has recently restricted almost all unauthenticated free access to their database while SecurityFocus’ BID database continues to have a fraction of the information they make available to paying customers. Quite simply, the price of aggregating and normalizing this data is high.

In the first case, we received a routine request for an account from a commercial security company, S21sec, that wanted to use our data to augment their services:

From: Marcos xxxxxx (xxxxxxx@s21sec.com)
To: moderators osvdb.org
Date: Thu, 16 May 2013 11:26:28 +0200
Subject: [OSVDB Mods] Request for account on OSVDB.org

Hello,

I’m working on e-Crime and Malware Research for S21Sec (www.s21sec.com), a lead IT Security company from Spain. I would like to obtain an API key to use in research of phishing cases we need to investigate phishing and compromised sites. We want to use tools like “cms-explorer” and create our own internal tools.

Regards,

S21sec

*Marcos xxxxxx*
/e-Crime///

Tlf: +34 902 222 521
http://www.s21sec.com , blog.s21sec.com

As with most requests like this, they received a form letter reply indicating that our commercial partner would be in touch to figure out licensing:

From: Brian Martin (brian opensecurityfoundation.org)
To: Marcos xxxxxx (xxxxxxx@s21sec.com)
Cc: RBS Sales (sales riskbasedsecurity.com)
Date: Thu, 16 May 2013 15:26:04 -0500 (CDT)
Subject: Re: [OSVDB Mods] Request for account on OSVDB.org

Marcos,

The use you describe is considered commercial by the Open Security
Foundation (OSF).

We have partnered with Risk Based Security (in the CC) to handle
commercial licensing. In addition to this, RBS provides a separate portal
with more robust features, including an expansive watch list capability,
as well as a considerably more powerful API and database export options.
The OSVDB API is very limited in the number of calls due to a wide variety
of abuse over the years, and also why the free exports are no longer
available. RBS also offers additional analysis of vulnerabilities
including more detailed technical notes on conditions for exploitation and
more.

[..]

Thanks,

Brian Martin
OSF / OSVDB

He came back pretty quickly saying that he had no budget for this, and didn’t even wait to get a price quote or discuss options:

From: Marcos xxxxxx (xxxxxxx@s21sec.com)
Date: Mon, May 20, 2013 at 10:55 AM
Subject: Re: [OSVDB Mods] Request for account on OSVDB.org
To: Brian Martin (brian opensecurityfoundation.org)
Cc: RBS Sales (sales riskbasedsecurity.com)

Thanks for the answer, but I have no budget to get the license.

We figured that was the end of it really. Instead, jump to today when we noticed someone scraping our data and trying to hide their tracks to a limited degree. Standard enumeration of our entries, but they were forging the user-agent:

88.84.65.5 – – [07/May/2014:09:37:06 -0500] “GET /show/osvdb/106231 HTTP/1.1″ 200 20415 “-” “Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0″
88.84.65.5 – – [07/May/2014:09:37:06 -0500] “GET /show/osvdb/106232 HTTP/1.1″ 200 20489 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko”
88.84.65.5 – – [07/May/2014:09:37:07 -0500] “GET /show/osvdb/106233 HTTP/1.1″ 200 20409 “-” “Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)”
88.84.65.5 – – [07/May/2014:09:37:08 -0500] “GET /show/osvdb/106235 HTTP/1.1″ 200 20463 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36″

Visiting that IP told us who it was:

s21-warn

So after requesting data, and hearing that it would require a commercial license, they figure they will just scrape the data and use it without paying. 3,600 accesses between 09:18:30 and 09:43:19.

In the second case, and substantially more offensive, is the case of security giant McAfee. They approached us last year about obtaining a commercial feed to our data that culminated in a one hour phone call with someone who ran an internal VDB there. On the call, we discussed our methodology and our data set. While we had superior numbers to any other solution, they were hung up on the fact that we weren’t fully automated. The fact that we did a lot of our process manually struck them as odd. In addition to that, we employed less people than they did to aggregate and maintain the data. McAfee couldn’t wrap their heads around this, saying there was “no way” we could maintain the data we do. We offered them a free 30 day trial to utilize our entire data set and to come back to us if they still thought it was lacking.

They didn’t even give it a try. Instead they walked away thinking our solution must be inferior. Jump to today…

161.69.163.20 – – [04/May/2014:07:22:14 -0500] “GET /90703 HTTP/1.1″ 200 6042 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36″
161.69.163.20 – – [04/May/2014:07:22:16 -0500] “GET /90704 HTTP/1.1″ 200 6040 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36″
161.69.163.20 – – [04/May/2014:07:22:18 -0500] “GET /90705 HTTP/1.1″ 200 6039 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36″
161.69.163.20 – – [04/May/2014:07:22:20 -0500] “GET /90706 HTTP/1.1″ 200 6052 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36″

They made 2,219 requests between 06:25:24 on May 4 and 21:18:26 on May 6. Excuse us, you clearly didn’t want to try our service back then. If you would like to give a shot then we kindly ask you to contact RBS so that you can do it using our API, customer portal, and/or exports as intended.

Overall, it is entirely frustrating and disappointing to see security companies who sell their services based on reputation and integrity, who claim to have ethics completely disregard them in favor of saving a buck.

mcafee-ethics

Cybercrime Stats: From Bad to Bad

Since vulnerabilities are a cornerstone of computer crime, stats on it are of interest to us. Statistics on cybercrime have always been dodgy; more so than real-world crime statistics. When your car is broken into or stolen, you know it. More often than not, you will report it to the police. In the computer world, an un-measurable number of intrusions happen every day. The rate of malware infection, DoS attacks, and other virtual crimes are not only difficult impossible to measure, they potentially go unreported more often than not.

Classically, the only number thrown around regarding damages from cybercrime has been this mythical one trillion dollars. Yes, with a ‘T’, not a ‘B’. That number has been challenged by many in the past, but no one has offered a better number with anything remotely factual. On July 22 the Center for Strategic and International Studies released a new study commissioned by McAfee (who previously quoted the trillion dollar figure) saying that damages are much less. From a Los Angeles Times article on the release:

Cyberattacks may be draining as much as $140 billion and half a million jobs from the U.S. economy each year, according to a new study that splashes water on a previous estimate of $1 trillion in annual losses.

“That’s our best guess,” said James Andrew Lewis, the director of the technology and public policy program at the Center for Strategic and International Studies.

James Andrew Lewis’ comment calling it a “best guess” is not assuring. The one trillion dollar figure cited for all those years was no better than a guess, as the surveys it relied on were far from a solid methodology. Regardless, the figure of $140 billion seems more rationale on the surface. Contrasting that is the claim that half a million jobs are “drained” from the U.S. economy each year. How can cybercrime conceivably lead to that? Reading on in the article:

Lewis and co-author Stewart Baker, a distinguished visiting fellow at CSIS, said that they were still working to determine cybercrime’s impact on innovation. They suggested a follow-up report might come out with a bigger number.

But preliminarily, they found U.S. losses to be somewhere between $20 billion to $140 billion, or about 1% of the nation’s GDP. They pegged job losses at 508,000.

“The effect of the net loss of jobs could be small, but if a good portion of these jobs were high-end manufacturing jobs that moved overseas because of intellectual property losses, the effect could be wide ranging,” Lewis said.

Right after the hint of a more rational number, CSIS immediately makes it a worthless number when they say it is really somewhere between $20 billion and $140 billion. In the world of sanity and statistics, that range is unreasonable. Further, Lewis goes on to say that some of the 508,000 jobs lost are due to “high-end manufacturing jobs moved overseas because of intellectual property losses”. Huh? High-end manufacturing jobs are moving overseas because of corporate budgets more than cybercrime. Such a claim should be backed up by a long list of examples showing companies losing intellectual property, and then reporting it to law enforcement or their shareholders, as well as SEC filings.

We moved from the fictional trillion number, to an overly wide range in the tens or hundreds of billions, and got an odd claim of half a million jobs lost due to cybercrime. This new study did little to clear things up.

@Stilgherrian makes another great point in his ZDNet piece, that everyone should take to heart when reading cybercrime statistics:

If we’re killing one cybercrime myth, let’s kill another — one which coincidentally emerged from McAfee — namely that the wealth transfer due to hacking represents some historically-unprecedented economic disaster.

Ultimately, we also have to remember that any cybercrime statistics coming from a company like McAfee are suspect, as they directly benefit them while they sell computer security solutions.

McAfee: Microsoft patches 133 Critical/Important Vulns in 2006

http://www.avertlabs.com/research/blog/?p=153

McAfee is reporting that Microsoft patched 133 Critical / Important vulnerabilities in 2006. They also compare this number against previous years to presumably demonstrate that security isn’t getting better at Microsoft.

Amusing Disclosure Timelines 101

We’ve all seen the standard disclosure timeline for a vulnerability. Date discovered, date reported to vendor, date patched, date disclosed. Once in a while, they are a bit more amusing.

http://archives.neohapsis.com/archives/bugtraq/2006-02/0543.html

McAfee notified 2006/02/17, denied responsability for the product and referred to Apple.
Apple notified 2006/02/17, denied responsability for the issue and referred to McAfee.
Published 2006/02/28 on Bugtraq.

Follow

Get every new post delivered to your Inbox.

Join 5,028 other followers