When clients ask us to help them develop their enterprise search strategy, we know that we’re probably going to have to dispel some of their expectations. A common belief is that there’s a universal magic search tool out there, and that the purpose of an enterprise search strategy is to select such a tool and define the rollout, whose future state will be everyone in the firm using the same tool to find any piece of desired content, from any relevant repository in the enterprise.
A good enterprise search strategy will address the different search needs across the enterprise, but it will not result in a single enterprise-wide search tool providing every employee the ability to search all repositories with a single, universal interface.
One reason is that there simply is no such tool – not from Google, Microsoft, IBM, Autonomy, Endeca, or any of the open source or pure-play specialists. We’ve been working with search technology and enterprise search strategies since the 1990s, and we’ve never seen this approach succeed. Most organizations who attempt this end up squandering time, money, and other resources which would be better directed at more effective content management and search activities.
Aside from the fact that the universal search tool organizations are looking for doesn’t exist, two other factors cause failure in enterprise search projects:
- Typically, enterprise search deployment roadmaps don’t define small stages that can provide wins at each step. Too much is left dependent on the success of big chunks of the rollout – chunks that are spaced far apart and that are at significant risk of delay or partial failure. As a result, the end users (the most important constituency of the enterprise search project) are disappointed throughout most of the rollout, and even if the implementation is technically successful, it’s often difficult to win back the trust of the thwarted searchers.
- Firms that focus their resources on enterprise search initiatives often neglect to move unmanaged and unconsolidated content into fewer and better-managed repositories, a step which can allow for easier and more efficient search. The result is the mess gets worse over the course of the search initiative. This is the point at which many organizations will then stop the search initiative in order start a management initiative. And as I’ve indicated above, most enterprise search initiatives don’t present “good” stopping points.
A good enterprise search strategy should address many issues, including the following:
- The range of your firm’s search requirements. Most of our clients – in many industries, including financial services, manufacturing, life sciences, and others – have a wide range of requirements across users and business units, except for the very low (simple) and very high (advanced) ends. The very simple search requirements are better addressed as “access” problems, and the very advanced requirements invariably fail any cost-benefit test.
- There are several different types of search. These include fielded keyword (indexed value) search, full-text search, document search, structured data search, web content search, search for e-discovery, and search for records management.
- The strategy should clarify the division of labor among your firm’s current search tools and any future enterprise search tools. Most organizations have several search tools already deployed, typically from Microsoft (Microsoft Search and possibly SharePoint Search), the search tools associated with their ECM solution (EMC/Documentum, IBM/FileNet, others), business applications, and some from search specialists.
- The ECM search strategy should coordinate with your firm’s e-discovery initiatives. Enterprise search and the “left-side” activities in e-discovery have many overlaps in requirements. Many firms buy redundant or unnecessary tools for e-discovery search, when they could get better bang for their buck by addressing other areas of e-discovery, such as legal hold management or early case assessment.
- The strategy should also focus attention and resources on moving unmanaged and unconsolidated content into fewer managed repositories, which can then more easily be searched by the systems’ own search capabilities and by any enterprise search tool your firm deploys.
Enterprise search solutions range from relatively basic tools to advanced platforms, with a generally corresponding range of complexity and resource requirements:
- Google, IBM, Microsoft, and several pure-play search vendors offer solutions at the lower end of complexity.
- Autonomy, IBM, Microsoft, and a few other vendors offer solutions at the higher end.
- The mid-range consists of products more advanced and expensive than SharePoint Search, but less advanced and expensive than Autonomy IDOL, IBM OmniFind Enterprise, or Microsoft FAST. It is occupied primarily by smaller or pure-play vendors such as dtSearch, ISYS, and SurfRay.
Since many firms’ search requirements probably range widely, except for the very low and very high ends, they have several options. All of them entail trade-offs:
- Implementing just a lower-end solution or just a higher-end solution. However, implementing just a lower-end solution (e.g. just SharePoint Search) will likely fail to address most of an organization’s search needs).
- Implementing just a higher-end solution (e.g. Autonomy IDOL) may be expensive “overkill” for many employees, whose desired documents will reside in a small number of managed repositories).
- The “Goldilocks” alternative then would be to implement a single product from the mid-range category. A mid-range tool would potentially meet all the needs of users with simple to moderate requirements and most of the needs of users with advanced requirements – and would potentially do so with lower resource requirements (cost, time, effort) than the advanced solutions. The major drawback of this approach, however, is that all of the mid-range solutions are provided by smaller or pure-play vendors, presenting significant vendor and product risk.
What often works is a small, mixed search portfolio – an approach that’s analogous to the mixed approaches for document management, workflow, and other ECM areas. Use the minimal set of products that both fits your various needs and makes the most sense from your firm’s current state portfolio. This may mean Microsoft FAST or Autonomy for advanced search and – for example – SharePoint 2010 Search (very different from generic Microsoft search tools) for more basic search.