Survey says: Don’t start with a bug bounty

This year, more than ever before, companies and organizations far beyond Silicon Valley have employed bug bounties to find security holes in their code. And according to a respected security company, their increasing reliance on hacker bounty hunters is costing them dearly.

In a report sponsored by computer code evaluation firm Veracode, Wakefield Research said that despite studies showing that the cost of fixing bugs skyrockets to more than 30 times the cost of fixing them after software has been released, most of the “500 IT decision makers working in cybersecurity” it surveyed rely heavily on bug bounties to catch security flaws.

READ MORE ON BUG BOUNTIES

Why Apple’s bug bounty is a big deal
Bug bounties break out beyond tech
The dark side of bug bounties

A large majority of survey respondents said they released code before testing for or fixing known security flaws, and they acknowledged that with proper developer training, those flaws could have been caught before the software’s release. More than a third said they relied on bug bounties to catch flaws after release, with 44 percent saying they’ve spent more than $1 million on bug bounty programs.

“What we tried to do is understand the relationship between bug bounties and internal efforts to secure software,” says Chris Wysopal, the chief technology officer and co-founder of software security evaluation firm Veracode. He compared it to doing food safety testing with a salmonella bounty. “The conclusion we drew is, don’t start with a bug bounty. That’s an expensive way to secure your software.”

Bug bounties represent a sea change in how computer code is determined to be safe for consumers to use. Software developers traditionally have hired firms like Veracode, often at a fixed annual rate, to regularly check their code for flaws before they release it to customers. They can run bug bounties, on the other hand, anytime before or after release.

Bug bounties have been an element of computer security since 1995. But in the past decade, a growing number of major tech firms, including Microsoft, Google, Facebook, Twitter, and LinkedIn, have adopted crowd-sourcing to augment their internal security measures. This year alone, Apple, Chrysler, MasterCard, and the Pentagon have launched bug bounties. And as with many tech trends, bug bounties have a dark side: There is a growing gray market for security vulnerabilities affecting the most popular software.

“The conclusion we drew is, don’t start with a bug bounty. That’s an expensive way to secure your software.” — Chris Wysopal, CTO and co-founder, Veracode

Despite the potentially troubling trends highlighted in the Wakefield-Veracode report, it remains unclear how often software companies are actually choosing to forego traditional pre-release security measures for bug bounties.

“I don’t know of anybody who has a bug bounty program and no [pre-release] application security program,” says Jeremiah Grossman, chief of security strategy for SentinelOne. “It’s disappointing there’s no metrics around that in the report.”

Grossman, founder of Veracode competitor WhiteHat Security and adviser to bug bounty manager Bugcrowd, has his fingers in both software security evaluation pies. He says that in terms of cost-effectiveness, bug bounties are most beneficial to “mature” software because many flaws plaguing untested or lightly tested code already have been fixed by earlier rounds of testing.

“The crowd-source model [used by bug bounties] will test you with a ton of tools,” he says. “When you crowd-source, you get blasted with everything, so you’re going to get more than what your internal team was going to find.”

Despite his interest in seeing traditional software security testing—or at least his own company—succeed, Wysopal says he believes that there are roles for both traditional testing and bug bounties in the future.

“Certainly, we support bug bounties and think they have a lot of value. What we tried to do is understand the relationship between bug bounties and internal efforts to secure software,” he says.

That relationship appears to potentially be more cooperative than merely competitive. In September, Bugcrowd partnered with NCC Group, a software-testing company, to offer public and private bug bounties to NCC Group’s corporate clients.