Real Story Group. Make Better Technology Decisions.

Kas Thomas

OASIS blesses UIMA - What does it mean?

20-Mar-2009

Tags: Enterprise Search, Industry Standards, Information Architecture, XML, , OmniFind Enterprise Edition

Yesterday, OASIS (the standards body behind things like DocBook and BPEL) approved a new standard, effictively resuscitating an arcane technology long thought by archaeologists to be dead: Unstructured Information Management Architecture (UIMA).

UIMA is one of those technologies that's hard to capture in a few words, but you can think of it as a pipeline-processing architecture and associated APIs for creating text-analysis components. Examples of such components might include tokenizers, summarizers, categorizers, parsers, stemmers, named-entity detectors, and so on. The oft-cited use case for UIMA is preprocessing of text destined for indexing by a search engine. But in fact, UIMA can be used in other contexts, as well. Any application that needs to parse or filter text, basically, is a candidate for UIMA (at least in theory).

IBM originally unveiled UIMA (to little fanfare) back in 2004, using it in the now-discontinued DB2 Intelligent Miner product before finding use for it in OmniFind Enterprise Edition (covered in our Search and Information Access Report 2009) and LanguageWare products (as well as, more recently, IBM's eDiscovery Analyzer). Except for a few users in government and academia, and one commercial firm (text-analytics vendor Nstein, which offers a dozen different UIMA "annotators"), UIMA never found traction outside the confines of Big Blue. Ultimately, in 2006, IBM open-sourced the technology and threw it over the wall (so to speak) to Apache, where it has been silently incubating ever since.

The significance of UIMA, technologically, is that it allows chunking of text-processing steps into units of work that can be implemented in different ways by different vendors, yet without losing interoperability. In theory, if your UIMA pipeline utilizes Vendor A's named-entity extractor today, but tomorrow you decide you want to use Vendor B's extractor because it's cheaper or better, you can simply swap out the respective annotators. (In UIMA, an annotator is like a plug-in filter.) You have a standardized pipeline architecture with Lego pieces (so to speak) that fit tightly together, instead of a jumble-bin of parts with randomly-sized square pegs and round holes.

The significance of the OASIS announcement (coming at this particular juncture) is hard to judge. It may be just one more data point (if any more were needed) validating the notion that interest in all things search-related is steadily rising. But it may also be an early indicator of interest in semantic-web technologies of the kind many (not all) experts feel will be important in the transition to Web.next-point-oh.

Regardless, it's the kind of "data point" we try not to overlook. So if UIMA begins to pick up steam now (finally, after five years) and land on a lot of other people's radars, don't say we didn't warn you.  We'll keep watching as part of our information access research, and share what we find with subscribers.