Real Story Group. Make Better Technology Decisions.

Alan Pelz-Sharpe

Enterprise Search - the best and worst of times

12-May-2011

Tags: Document Management (ECM), Enterprise Search, Marketplace at Large, Selecting Technology, CloudView, Coveo G2B, Enterprise Search, FAST and Search 2010, Google Search Appliance, IDOL Server, OmniFind Enterprise Edition, Vivisimo Velocity

I have been attending and speaking at the Enterprise Search Summit in New York this week, a good conference with many opportunities to rub shoulders and pick the brains of smart folk.

But it's also an event that gives me a sense of deja-vu, since so little appears to change in the world of search. In fact I am fairly convinced that I could pull out a presentation from a decade back, brush off the dust, and present it with a straight face today.

That is the problem with Search. So little seems to have changed, and in some regards little has.  To compound this problem, there is a somewhat justifiable perception that the Enterprise Search market is dominated by Autonomy, IBM, Microsoft, Google, and Lucene. That in short, the game is over. I say somewhat justifiable as there are many other vibrant and interesting search options to consider from Coveo, through Exalead, to Vivisimo. Similarly the key thing that has changed in the underlying search technology is that it has become ever more scalable, tweakable, and capable -- even if the basics remain familiar.

So if I were to mangle a sports analogy, I would argue that the first half of the game is over. We are in the interval and there is all to play for in the second half.

If the discussions at this conference and the reflections and insights from Real Story Group subscribers are anything to go by, there are a number of exciting trends in the search world:

1. Enterprise buyers are moving beyond the "Give me Googlelike search!" mentality
2. Search technology is going deeper under the covers and emerging ever more strongly via customized applications
3. Buyers are finally (if reluctantly) coming to terms that  "junk in - junk out" is as relevant today as its always been

Combine these three trends and we have enterprise search at an important inflection point.  We have at least a recognition that major search implementations need to be a part of a broader information management and organization effort (Point 3).  That employees already have a very sophisticated internal map of how their organization works, and where information is created and located. Therefore search UI's need to take a more balanced approached to navigational search that works in an organizations specific context, as opposed to a generic long list of potential "hits" (Point 1). And finally that people are interested in adding business value through analysis, not just searching for lost items. Hence the solid growth in diagnostic and analytical business applications that utilize in some measure search technology (Point 2).

So, for you as a corporate IT Director, what does this all mean in practical terms? Well it means a fundamental rethink of the value of search in your organization.  Recognize that the reason most people are searching for corporate information is because it isn't where its supposed to be. And that search can only ever be half of that conversation, information hygiene being the other half. 

Just as importantly recognize that in information-rich environments, search technology is able to deliver far more value than it typically does today, that positioning search as part of the "back-end" of highly specific and high value business analysis applications has the potential to deliver huge benefits.

When I think of how powerful search technology is today, and the tasks it is most commonly used for, I can't help but think of Marvin the paranoid android from the Hitchhiker's Guide to the Galaxy. "Here I am, brain the size of a planet, and they ask me to take you to the bridge. Call that job satisfaction, 'cause I don't."