Ever since I first looked into search technology about fifteen years ago, I've heard it was finished, productized, commoditized, and pretty much as good as it will ever get. Nothing new was to be expected. Every five years or so, I read a eulogy for search. Or at least, it's a "mature technology" that will see little improvement -- the story line goes -- and there are only one or two vendors even worth looking at.

It reminds me of a newspaper article I once read. It stated "the bicycle is now finished," because what could possibly be improved anymore? That was written in 1915, and yes, you can still buy bikes pretty similar to the bicycles sold back then. But you don't have to be a connoisseur to know you can get a lot more advanced bicycles nowadays. In fact, innovation seems to have accelerated in recent years.

Sure, the basic two-wheel concept seems to persist (though there's some outliers). You can tweak the wheels, make them more efficient, etcetera, but that's not usually where the most spectacular improvements are. For search, that's the inverted index: every product has one, and some are more efficient than others, but it's not an area that requires lots of engineering anymore. Indexing has been commoditized; in fact, some vendors, like IBM and Attivio, just use the open source Lucene (Solr) indexing engine. Why spend time reinventing the wheel?

However, the efforts go into the frame around it. There are plenty of ways enterprise search can be improved there. We now have bicycles for mountains, ice, city use, shopping, or speed racing. And a lot can be done to adapt search for specific scenarios as well -- or to make it more adaptable to individual implementations. And as with bicycles, innovation seems to have picked up the past two years.

Certainly, building your own implementation from a toolbox is too hard for most enterprises. That's where a lot of the "search is dead" is coming from. Disgruntled customers who've struggled through getting software "installed" that turned out to be a set of parts with more than the "some assembly required" they were promised. A sort of "Ikea Gøgle" that after a long struggle turns out to be lopsided and missing a screw. (At least Ikea has great manuals.)

But updating some of the product evaluations in our Search & Information Access research has been surprisingly exciting of late. Lots of things are happening in this space. And search is hardly a one horse race: different vendors have addressed different needs.

If you have to implement multi-repository enterprise search, you won't have to set a team of developers to work for years anymore. Each in their own way, Vivisimo and Coveo have made it easy to combine various sources in an implementation that is mostly configuration -- rather than assembly. However, if you do want to develop around search, infrastructure vendors like Microsoft have overhauled their framework -- making it a lot easier for developers to work with FAST in a familiar environment. And if you want sophisticated processing pipelines to create information dashboards, that's now actually becoming feasible, thanks to tools like Endeca's Lattitude and Exalead's mash-up builder. Open source (Lucene) Solr is quickly becoming an alternative to commercial vendors as an actual system (rather than just a Java library). And if you don't want to deal with any of this, Google is finally starting to live up to the expectations.

Those are just some of examples of what we describe in our Search & Information Access evaluations. There are plenty more. If, for the past few years, you have been thinking that enterprise search was stagnant, a mature technology developed to be a complex system, think again. Has search become a commodity? No. Indexing has become a commodity. But there are plenty of purposeful bicycles being built around those two wheels.