I had one of those midnight "wake up and go Doh!" moments last week. A common feature across nearly every conversation I've had about Enterprise 2.0 subjects hit me. Everybody says "Enterprise search is broken." In fact it is one of the first things to come up. But then people move on. As Churchill once said, people often stumble across the truth, but most pick themselves up and move on. I am guilty too. I first "stumbled" 3 years ago, and it's taken me this long to say, "wait a minute, I never thought that through."People move on because they seem to assume that this is incompetence at work. Search is sooo 1.0, right? It's been solved, and we're just fumbling the execution, right? You usually get some sort of ironic joke along the lines of "wow, it is so easy to find stuff out there on the public Web, and here with all our resources, we can't even do search right."And then the conversation tends to move on to more obviously "2.0" things like blogs, wikis, how to increase participation, and my personal pet peeve: annoying moaning about "culture change."Hold on. Rewind. Let's go back to search and think for a moment. I have a theory here, and I'd like to see if all you smart E 2.0 guys agree. I have reached a radical conclusion: broken search is the problem, but fixing search is not the solution. Search breaks behind the firewall for social, not technical reasons.How Search Breaks Behind the FirewallLet's start with the blindingly obvious, and then draw some weird conclusions. Here are the most common reasons that come up:
- Icebergs: Too much stuff that "ought to be shared" is on people's desktops, or sitting in emails in attachment form.
- Digital Landfills: There's still tons of those random fileservers all over the place, where people simply dump files. They "ought" to put it into content management systems.
- It ain't a net: Web-based content is just easier to crawl. You have all those nice, friendly sociable things like robots.txt files that tell crawlers where to go and not go. Intranets are much ruder. It's hard to take a proper inventory of everything that's not on the searchable "grid" even it is nominally "online" in some form. Corporate IT "ought" to clamp down on undisciplined asset sprawl.
- Those pesky formats: PDFs, PPTs, Excel spreadsheets, random proprietary databases with query interfaces.
- Permissions: Lots of need-to-know systems with non single-sign-on protections. No way you can give your friendly Intranet spider ALL the skeleton keys, right? Corporate IT "ought" to enforce corporation-wide single-sign on.
- Every single email ends up being an act of political judgment. To, cc and Bcc are three words about which I could write an entire book.
- Need to know and organization charts/cascade patterns beat democratic content popularity and "Word of Mouth" information travel hands down. This surprises people: we often remark about how much work gets done at the watercooler. But this does not mean more gets done that way than through formal channels. The formal channels carry 80% of the communication. The reason we focus on the watercooler is that most of the politically sensitive stuff travels that way.
- On the Web, people read people they like. Inside the firewall, everybody is constantly trying to figure out who's important, who has the money, whose stock is up, whose stock is down. Whose coat-tails to ride. Which fool to suffer gladly for the moment, which battle to pick. In other words there is very low correlation between information flows and friendship networks. You may pay most attention to people you hate or fear. Your best friend may be your lunch buddy from another department, and while you may be on each other's blogrolls outside work, you have no connection inside and don't vote up each other's information contributions. Oneworkgroup may be trying to stay under the radar in skunkworks mode, while another may have a reason for wanting a dog-and-pony influence roadshow this year.