Think of Splunk as Google for systems administrators, a search tool that looks inward instead of out.
The latest version of the 9-month-old tool, released last week, hunts down problems found in log data culled from companies' IT systems. And Splunk 2.1 adds a feature that mimics another Google practice: furnishing command line APIs to the search engine so third parties can write independent applications that exploit its capabilities.
LogLogic, LogRhythm, and EMC's Network Intelligence offer competing products that collect logged data, but they're positioned more as instruments that help companies capture server information needed to comply with industry standards and government regulations rather than troubleshooting tools.
As developers independently write applications that incorporate Google, Yahoo, and other search engines into applications for the Web, such as mapping apps, software writers can use Splunk's APIs to create, for instance, a Flash application that monitors real-time security threats and visually pinpoints the source of an attack. Another possible use of Splunk's search capability would be as a visual tool for a marketer that depicts major sites blocking its outbound e-mail, as indexed in the marketer's internal IT systems logs.
The new Splunk release can run on multiple servers, with search results merged and presented in an interactive Ajax Web user interface. It requires 40% less storage capacity for indexing and storing original log and IT data than the last version. Splunk CEO Michael Baum contends the tool is up to five times faster than conventional log technologies and log appliances and can work more quickly if needed since it can run on clustered servers. Indexing speeds range from 20,000 to 120,000 events per second on a single server.
Pricing is based on the peak daily volume, starting at $2,500 a year for indexing 512 Mbytes of raw, uncompressed data each day. It scales to 1,000 Gbytes for $300,000 a year.