Opinion: Enterprise Search Is This Year's Hot Topic For Business Intelligence

Will enterprise search become part of the business intelligence sector of enterprise IT? Plenty of analysts and experts said 'yes' at last week's Enterprise Search Summit, yet many practitioners have yet to forge a real search strategy. Here's a bit of advice on how to make the most of current investments and how to choose a new search platform.

Last week I soaked in the latest updates from the world of enterprise Web technology at the Enterprise Search Summit in New York City. Based on what I heard, no one seems to know for sure what enterprise search will look like in the next few years. But everyone agrees on one thing: It's hot.

Chaos and Confusion

Susan Feldman, Research VP of Content Technologies, IDC, opened the event with an aptly titled presentation, "Search in 2007: Growth, Chaos, & Confusion." Feldman opened up with the obvious: Search is growing like a weed. The search market last year grew 33 percent. The problem is that no one seems to know what this growth means for enterprise IT departments or even for the search market itself, with the exception that Google is still the king of the consumer search market.

In an attempt to give some thought leadership, Feldman argued that search is rapidly becoming a part of the business intelligence (BI) sector of enterprise IT. This isn't the first time I have heard this argument, and it seems to be playing out as more search vendors position their products as BI solutions and not just search platforms. Feldman also pointed out that search is a highly political issue in many organizations, both because it's trendy and because it's still a category in flux. Everyone inside a large company wants to own something and search is up for grabs. IT has a vested interest in owning it, but so too do other departments, especially online marketing.

Trying To Make It Work With Your Current Search Engine

After the opening keynote, the discussion turned to more practical issues: Should you dump your search engine or try to stick it out?

Jennifer Whalen, Portal Manager for Continuing Improvement, Deloitte, argued that many organizations are probably better off trying to improve and optimize their current search engines than in looking for new technology. Whalen pointed that many search systems suffer both from slack technology as well as poor content. Over tagging -- i.e. tagging too many pieces of content with the same tags -- as well as poorly sorted content can lead to bad search. Many companies improve their search by cleaning up their content. Another key is testing. Few organizations spend much money or effort in testing their existing systems. Instead of just dropping tons of cash on a new system, organizations can also test their systems and use these tests as ways to benchmark and improve performance.

There are many ways to make search results more relevant. It's not always wise to focus on the long tail of search results at first. First, make the majority of common searches work and then expand beyond. Whalen advised IT managers to create lists of 30 or so sample queries and then to optimize around those results. IT managers should also focus on making their existing search engines easier to use. Simple improvements in UI can also increase the relevancy of search results.

It's Time To Go Shopping

Many of the attendees at last week's events, however, had little or no existing search solution. They were there to examine new technologies and spend some money. BI analyst Theresa Regli, Principal, CMS Watch, offered the attendees some advice on how to intelligently shop for a search solution. Regli cautioned against the "all-in-one" solution, "The ultimate knowledge management machine simply does not exist. The typical enterprise search system does not contain 'all' the organization's content." And in many instances, it's not wise to include all the content.

As for what makes one search engine different from another, Regli laid out a list of differentiators:

  1. Clustering Results and Related Terms -- Can the search platform organize results into broad topics?
  2. Metasearch -- Can the engine search across multiple disparate databases?
  3. Content Types -- Can the engine search across different types of content, such as Word documents, e-mails, database tables and XML/HTML?
Most organizations are currently debating between three search solutions:
  1. Local Install
  2. Hosted
  3. Appliance (search in a box)

There are distinct advantages to each solution. Regli advised against a local install if the organization isn't willing or doesn't have the resources to upgrade and maintain its investment.

Several themes emerged from the event speakers. The first was that search is now a form of enterprise BI. If this proves true, you can look for search to merge with other BI solutions, maybe even going back into content management systems themselves.

The second is that enterprises want search, but they're uncertain what they want from it besides more accurate search results. This indicates to me that many IT departments don't have a strategic view of search. Instead they see search as a panacea that will cure their online initiatives.

The third theme was content. From poorly managed content to bad tagging, there is a lot of badly sorted online content out there. Many businesses don't seem to have online content strategies, and I suspect that the lack of search strategy comes out of this. Stephen Wellman is director of newsletters for CMP's Business Technology Group and Editor of the "Grok on Google" newsletter.