Doculabs

 
There is too much information on the Internet. To make matters worse, there is too much bad information on the Internet. (I don’t mean “morally reprehensible,” although that is true as well.) Search engines are trying to direct us through the worldwide swamp of content out there. But this approach is based on algorithms that can be gamed. SEO, anyone?

I want to examine the problem of Internet search to make some suggestions about how enterprise search could be improved.

Commercial Search

The problem, as I see it, is more philosophical than technological. The philosophy behind our modern methods of search is that you should be able to find anything. The search engine will help you find things that lots of other people have found interesting, but ultimately the entire Internet is being searched. The only way to establish “value” in this view is based on volume of traffic or links to specific content.

Maybe I’m a little bit of an information snob, but I don’t care what the most popular sites are on any given topic. I want the best information. And when I say “best,” I mean I want information that people who are respected in their field either agree with or are producing. I could use a “site:” search and limit my inquiry to newyorker.com, for example. But what about writers and thinkers who are like the writers and thinkers on The New Yorker web site? How am I supposed to find their work among the thousands of blogs and content farms that are proliferating?

This is something that an algorithm can’t give me. Not today, anyway. It’s also not something the search companies want to give me, because they want to give me the ability to find anything, regardless of “value”.

This is where I think we need the super nerds. Not the algorithm-writing nerds – the other nerds. Economics, English, Theatre, Journalism, History, Theology, etc. – those nerds.  Basically, I want an army of super nerds who will act as curators of online content. I could go to their site and search for anything on my mind and instead of getting a list of hundreds of millions of search results, I’ll get only a few. Maybe I won’t like those results. That’s okay; Google isn’t going anywhere. But the super nerds would give me the option to have an edited version of the Internet. Call it a New Yorker-style web that I can go to and know that everything I read is academically insightful, interesting, and current in the conversation of the field in which it occurs.

Enterprise Search

There are super nerds in your organization as well. I say enterprise search needs to be more than just the search tools that are bought and installed for organizations. Search strategies should find ways to assign value to the super nerds in each division of the organization (Sales, Accounting, Operations, etc). The content those individuals create and find useful is likely to be more important than the rest of the departmental sludge that your search engine will churn through. For example, Richard is always creating useful information about our accounting processes. When other individuals in the accounting program do a search, Richard’s information should show up higher in the results because he creates more valuable content.   

The wrong way to do this would be to try to assign a value to every piece of content that’s created. Instead, persons of authority – either department heads or knowledge brokers in a department – would be given editorial rights to tag a specific piece of information as useful. Alternatively, they could assign a certain user a higher value in the content creation scale so that everything created by that user would appear higher in search results. 

That’s how search would work in my dream world. I’d love to hear your comments.