5 min read

Search is Not a Problem

The BrainYard - Where collaborative minds congregate.

I have noticed it is becoming popular again for vendors selling search solutions to quote surveys showing the average worker spending 15%-30% of their time looking for information. This is always, of course, cast as a problem that search can solve.

To me, these huge numbers point to fundamentally larger problems.

This is a symptom of poor Intranets that make it difficult to get your work done. Just because we cannot find something quickly does not mean search is the answer. It is like telling a teenager that the real reason they cannot find a particular shirt is because they are unable to search quick enough through the mounds of clothing cast across their bedroom. A person needs to know what clothes they have, what clothes are clean, and then balance that against their need for clothing. Perhaps it makes for a simpler file cabinet or vault but an approach based on search technologies does not make for an effective workplace.

It is more likely we try to improve our productivity by avoiding search altogether. Isn't search the last "Hail Mary" pass we put up in the hopes that we can find a thread that will get us going? Certainly, we are relying on search more and more as evidenced by recent studies. Is search the new standard application interface or is it the standard fallback method used when we are lost or confused? Perhaps we lost so much faith in our applications that we just assume search is the best place to start since we end up using it so often anyway.

I think the bigger problem is the lack of focus on the total Intranet experience. Most of us in IT like to deal with things we can manage, knobs we can turn, and systems we can tune. Search technologies fit IT perfectly.

How much of the 15-30% wasted time is attributable to searching? I suspect a significant portion of it can be credited to the overhead we force people to go through to get the information they need. This involves the numerous fragmented systems that are masters of their own data, commanding separate user interfaces and access control models. These applications are rulers of their own little fiefdoms, unrelenting in giving up control to the masses who consume their data. You got a problem with that? Well then log a ticket with the help desk.

These are definitely not search problems; these issues deal with ease of information access. The first information aggregator in the world was not using technologies like RSS, ATOM, or XML. The original aggregator is the poor information worker who keeps three-ring binders full of printed information so they do not have to login to the multitudes of enterprise information systems just to get a small task completed. Their workplace is their desktop; the one upon which the computer sits, the one you can knock with your knuckles (or pound with your fist as the situation warrants), not the computer's desktop interface.

At the technological root of this problem is a need for lightweight access to information that we process every day. This information should be intelligently aggregated in a way that makes sense to the individual. Certainly collaborative environments contain much of this information, perhaps in the form of documents or lists. Collaborative environments may even be one of the places where this information should be aggregated. However, we also need information from many other sources including enterprise systems and information accessible through the Internet.

As Intranet strategists we need to encourage application architects and developers to add flexible, lightweight XML interfaces that can be transformed into formats such as RSS, ATOM, and others that can be consumed by the end-user in systems like news aggregators, Google OneBox, Microsoft's SharePoint Business Data Catalog, etc.. The challenge for many applications will be to provide data in an XML format since it has been locked up in closed systems for so long. Once applications get data to that point transformation into the appropriate format should be straightforward. The challenge may be to convince management that these lightweight interfaces are necessary to enable the new workplace. Most web services have been put in for specific applications, not for something as mushy as well...productivity.

Paul Kedrosky wrote about this in 2005 when he said:

All that information is out there and useful to someone, even if it's someone waaaay out in the long tail (you knew I was going to say that). But it's not been cost-effective to expose it. Until now. …What would you (or your organization) monitor if you could? What would like to know about that you don't? What happens on a daily basis that would like to track but you can't? What if you could search all that information prospectively and retrospectively? It's a world-changer.

This is a great quote since it recognizes that most small pieces of information are valuable to only specific individuals. Many of you have conquered the big information distribution challenges. Now it is time to get personal and conquer the small pieces of information that needs to be distributed in a thousand different directions. If it follows the trend of other long tails we may find these small pieces of information make up most of the need.