Developers' Tool Improves Open Source Security, Trims Defects
Several software teams consider Coverity's Prevent SQS a valuable product despite a number of false positives.
Source code scans, such as those being performed on open source projects for the Department of Homeland Security, have become an important new tool in eliminating bugs at many of the projects, open source developers say.
At the same time, the scans are illustrating that average open source projects are, well, average, when it comes to creating bugs. Commercial code and some of the open source projects under review are showing one code defect or security exposure per 1,000 lines of code. But the best projects are showing a defect rate far lower than that. They're also illustrating how open source, unlike some commercial teams, is willing to air its defects and clean them up quickly.
"We log into their Web site each week and see what they've found," said Jeremy Allison, lead developer and co-founder of the Samba open source project. The code scanner, Coverity's Prevent SQS, "has got an inhuman eye for detail. It's like having the most persnickety programmer in the world looking over your shoulder," Allison said in an interview. Samba is included with most distributions of Linux and converts a Linux server into a machine that can talk with Windows.
The Samba team has fixed 228 bugs found by Prevent SQS and still has 11 findings to review and determine whether they're real bugs. The team inspects each finding because Prevent SQS returns false positives, and Samba developers notify Coverity when an alleged bug is actually good code so it can improve the tool's scanning ability.
"Their false-positive rate is low enough for it to be an extremely valuable tool," Allison added. The findings were complicated by the fact that Samba switched its change management system from Subversion to GIT, and broke the scripts that Coverity used to download the most recent builds or compilations of Samba. For a while, no bugs showed up because Prevent was rescanning previously fixed code. Then it reached the right repository and many days of new development showed up with a bunch of new defects.
But on the whole, "I was quite pleased with what Coverity said about us," Allison said. The scanning results show Samba with a defect rate of 0.024 per 1,000 lines of code instead of the average 1 per 1,000.
Not everybody has felt that way, as results of the scans have been aired by InformationWeek. "This story is just free and open source software bashing," said a reader in a submitted comment. Many readers wanted to see a comparison of open source to commercial code, but proprietary software companies are secretive about their defect rates. "Seems seriously slanted," said a commenter. "Steve Ballmer, is that you?" said another.
[Interop ITX 2017] State Of DevOps ReportThe DevOps movement brings application development and infrastructure operations together to increase efficiency and deploy applications more quickly. But embracing DevOps means making significant cultural, organizational, and technological changes. This research report will examine how and why IT organizations are adopting DevOps methodologies, the effects on their staff and processes, and the tools they are utilizing for the best results.
2017 State of IT ReportIn today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.