Startup Supports Lucene Open-Source Search

For the most recent edition of the Search and Information Access Report, I had the opportunity to help expand our coverage of Apache Lucene. The decision to increase our coverage of Lucene was a no-brainer, because (as anyone covering the search and information access space knows) Lucene's traction in the enterprise world is accelerating by the day.

Over the past year, in particular, interest in Lucene (and related projects - Solr in particular), has picked up noticeably, in part because of recent significant performance enhancements and stability improvements, but also because IT specialists are increasingly desperate to get their search problems solved on a limited budget.Lucene has a lot going for it (including a .Net version that seems to be gaining momentum). As open source projects go, it's one of the safest ones around in terms of governance and oversight (Apache Foundation), the maturity of the code, the amount of active development going on, the size and vitality of the user ecosystem, and the number of high-traffic websites that have validated the technology in real-world applications (some better-known examples being Monster.com, Netflix, and Wikipedia). Perhaps reflective of all this, Lucene has become a top-five Apache project, with 7,000 downloads a day.

But one thing Lucene is not is an out-of-the-box solution. It's strictly a code library. And it's just for indexing and querying. It has no front end, no Web Services layer, no spiders, no document-level security. To go from Lucene to a ready-to-deploy solution requires programming (and lots of it). And when you have a problem, there's no phone number to dial in the middle of the night. It's just you, the source code, and the community.

The lack of commercially available support and training have been a bit of a hindrance to acceptance of Lucene in the enterprise, but that's starting to change. This week, fledgling services company Lucid Imagination officially arrives on the scene to offer commercial-grade support, formal developer training, and certified, tested versions of Lucene (similar to the way Alkacon Software operates as the commercial entity behind OpenCMS). What makes Lucid Imagination unique is its connection with so many Lucene luminaries. The founders include Yonik Seeley (creator of Solr), Grant Ingersoll (Lucene/Solr committer and PMC member), Erik Hatcher (Lucene committer and author of the Lucene in Action book), and Mark Krellenstein (former CTO of research portal company Northern Light). Doug Cutting, the creator of Lucene, is on the company's Advisory Board. [Editor's Note: There's no apparent connection between Lucid Imagination and SaaS-based BI vendor LucidEra]

Lucid Imagination intends to offer its own certified versions of Lucene, which will simply be the most current Apache release (not a fork), tested and debugged by Lucid Imagination (with bug fixes donated back to Apache). Support subscriptions are offered at three service levels, the most expensive of which (with an advertised 4-hour response time) runs around $18k per year. Eventually the company says it plans to offer a developer certification program, but for now the emphasis is on support contracts.

Whether Lucid Imagination (which received $6 million of Series-A funding this week) succeeds over the long term or not , the arrival of commercial support signals the passing of yet another important milestone for Lucene. It tends to underscore the notion (if it needed underscoring) that Lucene has graduated from open-source curiosity to serious enterprise tool.The only question now is how far the community will take it in the form of ready-to-deploy applications built on Lucene. That story, I think, is just beginning.The lack of commercially available support and training have been a bit of a hindrance to acceptance of the open-source Apache Lucene search engine in the enterprise, but that's starting to change. This week, fledgling services company Lucid Imagination officially arrives on the scene to offer commercial-grade support, formal developer training, and certified, tested versions of Lucene.