Real Story Group. Make Better Technology Decisions.

Apoorv Durga

Follow Apoorv on Twitter @apoorv

IBM Search, powered by Lucene, is a forklift upgrade

21-Sep-2010

Tags: Enterprise Search, Marketplace at Large, Open Source, Selecting Technology, OmniFind Enterprise Edition

Rather quietly, IBM has replaced the search index within their OmniFind Enterprise Edition. The latest release of this search engine, version 9.1 supports new search collections that are based on Lucene, something about which we had blogged  earlier. The search collections in earlier versions are still supported, but are now called "Classic Search Collections." Depending on whether you do a complete new install or upgrade an existing installation, you will have access to only the new Lucene based-collections or both the new collections as well as the classic ones.

This switch has not received as much attention as it deserves. If you are using classic search collections, you can keep using them but you will certainly need to consider migrating them to the newer search collection even if you don't want to take advantage of the new features. This is because IBM has deprecated the classic search collections. IBM will also not be supporting the existing Search and Index APIs (SIAPI admin APIs) as well as the Web Services admin APIs. This is because Lucene has its own new REST-based API. So if you have your own applications, you will need to change them or create new ones that use the new REST API.

So let's get this straight. The new version brings:

• A completely new index -- which means re-building your entire index, with potentially unexpected results, and
• A completely new API -- which means any application development or advanced customization needs to be re-done

Sounds to me more like a replacement than an upgrade.  Though you still get legacy support.

On the plus side, the new version also makes it optional to use WebSphere Application Server and you can chose to use Jetty instead of WebSphere if you prefer a more lightweight application server  -- and many do.

There are many other important technical implications as well as potential benefits that we'll cover in the Search and Information Access Research. In the meantime, it's certainly a big news for Lucene. IBM has been a regular contributor to Lucene and this move shows their endorsement of Lucene's maturity.  One of IBM's other search offerings, OmniFind Yahoo Edition (OYE), was already built on top of Lucene, though it remains to be seen if IBM takes some steps to rationalize the two offerings.