On the Value of Automated Code Scanners
Posted by jericho
CodeScan Labs recently disclosed that their new product was used on ASP Portal to look for vulnerabilities. These types of scanners are automated and check for common programming errors that lead to vulnerabilities. These types of tools have been around for many years, but are starting to mature quickly. However, one has to wonder just how effective they can be:
2006-03-02 - ASP Portal announces version 3.1.0 which contains “CodeScan security fixes” 2006-03-03- ASP Portal announces version 3.1.1 which contains “a critical security Fix” (in news_item.asp) 2006-03-14 - CodeScan discloses their tool found 10 SQL injections and over 50 cross-site scripting vulns 2006-03-20 - nukedx releases a working exploit for an SQL injection (in download_click.asp) 2006-03-21 - nukedx releases details for 10 sql injections in 3.1.1 including one in news_item.asp
So CodeScan finds 10 SQL injections, but doesn’t find the 11 others that nukedx finds a week later, and doesn’t find the “critical” issue in news_item.asp either. Hopefully these tools continue to mature very quickly. Maybe some day, cross-site scripting vulnerabilities will be a thing of the past! Hah yah right, if that were true, overflows and race conditions wouldn’t pop up every few days either.
The problem with even the best code scanner is that a lack of findings doesn’t mean there aren’t any vulnerabilities–just that the tool didn’t find any. Still–automated code checking is still better than none at all!
The other way to look at it is that the tool found 10 or more vulnerabilities that were previously undisclosed. By getting these fixed and educating the developers, hopefully down the line they will introduce fewer bugs. Hopefully.
When we first contacted ASP Portal about the issues we found with our CodeScan ASP source code scanner we reported a total of 30 unique SQL injection flaws. These flaws were in code that used user supplied input without passing the input through any filtering function. We supplied a detailed list of these issues, and an exploit POC for one of them.
We also alerted them to a flaw in their filtering function whereby numeric type input was not filtered at all. This leads to approximately 84 more unique cases of SQL injection, authough these were not individually confirmed.
After the 2006-03-02 release, we downloaded the new version and were shocked to see that while they had fixed the issue that we supplied the POC for, they had not fixed all the others and had not taken our advice about the filtering function either.
We contacted the vendor again, providing another exploit POC and stressing the fact that they need to fix the root cause of these issues. They then released the 2006-03-03 Critical Security Fix. This time we were not really that suprised to see that they had only fixed the issue that we had provided the POC for.
A few more emails went back and forth as we tried to convince them to fix all the problems, but at the end of the day there was no way we could force the vendor to do this.
When we released our advisory, we realised that all the issues we discovered were not fixed and worded the advisory accordingly;
“… a new version of the software has been released to address a number of the discovered vulnerabilities.”
In accordance with our responsible disclosure policy, we were not able to release full details on all our findings, and realised that both CodeScan and ASPPortal would be targetted post advisory release.
We are confident that our CodeScan ASP source code scanning tool found all the ‘currently disclosed’ SQL vulnerabilities, and are in a position to provide the original report details to jericho for independent confirmation.