Sourcefire Slams Open Source IDS Suricata

Matt Olney, a senior research engineer for the Sourcefire Vulnerability Research Team (VRT), recently launched a war of words against Suricata, the new open source IDS.

Developed by the Open Information Security Foundation (OISF) -- funded by the Navy's space and warfare command, security vendors, and the DHS -- Suricata has been billed as a way to "bring new ideas and technologies" to IDS. It uses the same rule set as Snort, but according to the OISF also offers "other capabilities above and beyond the standard Snort rulesets."

Snort, by some accounts the world's most-used intrusion detection system (IDS), is maintained by Sourcefire, which also provides commercial services and support for enterprise Snort users. The day Suricata debuted, Sourcefire's stock price took a dive, though it's since recovered.

Suricata, of course, is also designed, like Snort, to help companies spot attacks, and in a Sourcefire blog post, Olney said he'd welcomed the innovation that Suricata first promised. Since then, however, his attitude has changed.

"Having worked with Suricata and looked at what the OISF has actually delivered, I'm just disappointed with where they've ended up and what they've delivered," he wrote. For example, Suricata emphasizes its use of a multi-threaded architecture running on commodity devices, but Olney says there are sound reasons for not doing this. "Trust me, if multi-threading were the answer, the industry would have moved there in short order."

Olney also released the results of an internal test pitting Snort against Suricata, with both optimized for maximum performance. According to Olney, "With rules loaded, Suricata runs up to about 200MB per second. Snort, with rules, hits 894MB per second with no drops."

But according to Matt Jonkman, president of the OISF, the decision to go multithreaded was made after extensive tests by the Air Force Research Labs.

Furthermore, he disputed the Sourcefire performance tests. "Those stats are ridiculous, and they refuse to publish" details of the equipment and configuration used, said Jonkman. "We know that we're not, right now, cycle for cycle, faster than Snort … but we're getting six times the performance as Snort on the same hardware, with version 1.0." Version 1.01 was released yesterday.

"We're not a finished engine, this is a 1.0 release, and we have a ways to go with optimization and accuracy," he said, also conceding that building an IDS from scratch was not for the faint of heart. "It was 10 times harder than the worst case we thought about."

But the initial goal, he said, had been to get Sourcefire to "shed the burden of developing" Snort, and let the OISF maintain the code base. "It's a very good engine, good at what it does, but it's not what it used to be." Furthermore, he said, while anyone can examine the Snort code, Sourcefire no longer allows any code contributions, and doesn't release a bug tracker.

Accordingly, he said, the DHS funded the OISF "because they felt the community needed a truly open platform to gather around. They wanted to see something new, and kick start innovation."

Olney, meanwhile, has said that if the market wants innovation, then it should look no further than Razorback, a new, real-time and open source analysis and detection engine, released last week by Sourcefire. "It isn't Snort, it isn't ClamAV, and it isn't Suricata," he said. "It's a new approach to the detection problem, and was built from the ground up in close collaboration with groups that are facing [advanced] threats. It may not be perfect, it may not even be the right answer -- but we think it is -- but it is truly innovative."