Enterprise Search Is This Year's Hot Topic For Business Intelligence

This morning I soaked in the latest updates from the world of enterprise Web technology at the packed-out Enterprise Search Summit in New York City. Based on what I heard, no one seems to know for sure what enterprise search will look like in the next few years. But everyone agrees on one thing: It's hot.Chaos & Confusion

Susan Feldman, Research VP of Content Technologies, IDC, opened the event with an aptly titled presentation, "Search in 2007: Growth, Chaos, & Confusion."

Feldman opened up with the obvious: Search is growing like a weed. The search market last year grew 33%. The problem is that no one seems to know what this growth means for enterprise IT departments or even for the search market itself, with one exception: Google is still the king of the Internet-based consumer search.

In an attempt to give some thought leadership, Feldman argued that search is rapidly becoming a part of the business intelligence sector of enterprise IT. This isn't the first time I have heard this argument, and it seems to be playing out as more search vendors position their products as BI solutions and not just search platforms.

Feldman also pointed out that search is a highly political issue in many organizations, both because it's trendy and because it's still a category in flux. Everyone inside a large company wants to own something and search is up for grabs. IT has a vested interest in owning it, but so too do other departments, especially online marketing.

Trying To Make It Work With Your Current Search Engine

After the opening keynote, the discussion turned to a more practical issue: Should you dump your search engine or try to stick it out?

Jennifer Whalen, Portal Manager for Continuing Improvement, Deloitte, argued that many organizations are probably better off trying to improve and optimize their current search engines than in looking for new technology.

Whalen pointed that many search systems suffer both from slack technology as well as poor content. Over tagging -- i.e. tagging too many pieces of content with the same tags -- as well as poorly sorted content can lead to bad search. Many companies improve their search efforts by just focusing on cleaning up their content.

Another key is testing. Few organizations spend much money or effort in testing their existing systems. Instead of just dropping tons of cash on a new system, organizations can also test their systems and use these tests as ways to benchmark and improve performance.

There are many ways to make search results more relevant. It's not always wise to focus on the long tail of search results. First, make the majority of common searches work and then expand beyond that. Whalen advised IT managers to create lists of 30 or so sample queries and then to optimize around those results.

IT managers should also focus on making their existing search engines easier to use. Simple improvements in UI can also increase the relevancy of search results.

It's Time To Go Shopping

Many of the attendees at today's events, however, had little to no existing search solution. They were there to examine new technologies and spend some money. BI analyst Theresa Regli, Principal, CMS Watch, offered the attendees some advice on how to intelligently shop for a search solution.

Regli cautioned against the "all-in-one" solution, "The ultimate knowledge management machine simply does not exist. The typical enterprise search system does not contain 'all' the organization's content," Regli said.

As for what makes one search engine different from another, Regli laid out a list of differentiators: