The Fast Search & Transfer news continues to keep me on my toes. The Norwegian business weekly Dagens Næringsliv seems to have come up with some decent evidence (English translation) of many things everybody already suspected -- and a couple of new ones.

It covers the Convera acquisition and the "separate" but surprisingly coincidental deal where Convera bought several million's worth of Fast software it didn't need. Or as rival Autonomy (which was also in the running for buying Convera) has pointed out, Fast pumped up its revenues for that quarter with part of the money it paid for Convera, then got back for licenses. The DN article also covers a few other very suspicious deals, and some outright fraud. It's now even getting to the point where calling Fast "the Enron of Norway" is getting long in the tooth.

While that train wreck was unfolding before my eyes in slow motion, my fellow analyst Theresa Regli pinged me last February about a new enterprise search company called Attivio. Information Today raved about their new product AIE, with analysts quoted as saying things like "they are moving rapidly to develop tools that will eliminate many of the practical barriers to easily and efficiently deploy robust enterprise search solutions," with the unique selling point of "data integration plus search and content processing," a "hot niche for the next few years."

Since I'm always interested to find out more about robust enterprise search tools to fill hot niches for the next few years, I scrolled down to read what the Attivio CTO would explain about how the product would achieve what "should have been solved by the integration of text search and XML into relational database managers such as Oracle." As it turns out, it is based on a "mash-up" of open source Apache Lucene and "licensed commercial software."

As described in our Enterprise Search Report and on this blog, Lucene itself is just a Java text search API. To be able to actually gather, convert, and query content you need many more components. It is perfectly feasible to put together a working enterprise search product around the core Lucene JAR (as demonstrated by IBM's Omnifind Yahoo! Edition). But in order to get there, and to have Lucene index, for instance, Office documents and PDFs, you will have to first convert those documents to text. The filters to perform that conversion can be bought from other vendors, based on open source such as pdftohtml, or you'll have to build them yourself, which is a lot of work. There aren't too many vendors building their own filters, or even just modifying open source to do so. So if you do build the filters needed to use Lucene yourself, you'd probably like to mention this as an advantage, and as Attivio states, "we developed our own Microsoft Office, WordPerfect, and PDF connectors to improve performance and reach deeper into the files than the conventional converters."

Since, like most enterprise search products, Lucene isn't based on a database and couldn't even connect to such content without help, it isn't surprising Attivio had to develop a "unique RDBMS data loader" which "indexes the tables individually." This, again, is presented as a major advantage -- remember, converting documents and integrating structured and unstructured data are "a hot niche."

I remember seeing a vendor at a conference a few years back, with banners jokingly stating its product was "buzzword compliant!" Attivio certainly seems to have that skill down. The engineering effort is marketed as a "technology mashup," "breaking down silos" between "open source and commercial software." "We have lived with the challenge of having to choose between the precision of databases and the richness of search for a long time, but no longer" sounds great, but I don't see Oracle and Thunderstone's RDBMS-based solutions breaking out in a sweat just yet.

Maybe my over-exposure to marketing materials and flashy demos has turned me into a cynic, and Attivio's downloadable trial version will have to do at least a decent job to convince me of the product's added value. Fortunately, that free download is "coming soon!" Yes, I'm sorry, I'm finding it increasingly hard to turn off that cynicism, especially when I turn back to the DN article about Fast Search & Transfer. Attivio was founded by former Fast employees and the Attivio CTO is Sid Probstein, formerly vice president of technology at FAST. More importantly, Attivio's CEO is Ali Riaz, who was COO at Fast but unexpectedly left the company in late 2006. Well, in hindsight, perhaps not so unexpectedly, though DN quotes him as saying "I had nothing to gain from manipulation of the accounts. I had no shares in the company. I wanted shares and quit because I didn't get any. If you want to find out what's wrong with the accounts, you need to look at those who could gain from it. And it wasn't me."

Dagens Næringsliv doesn't appear to agree with Riaz, however; if you want the full analysis of the what and why, I suggest you read the article. I myself find it surprising that the CEO of a technology startup backed by $6.2 million in venture capital would drive an Audi R8, but that doesn't mean anything (other than that I'm envious of his car). I also find it surprising a former Fast COO would be co-owner of a company reselling Fast licenses, but walking like an Enron duck and quacking like an Enron duck doesn't necessarily mean that it's really anything like Enron. And Attivio's clouding the core technology in marketing hyperboles and buzzword compliance is slightly disconcerting, but many renowned companies engage in the same practice.

DN quotes Riaz as saying "you should be much better at praising the people who have success, instead of pushing them down." And I would certainly love to be proven wrong by Attivio's software; as soon as I get my hands on the trial download I requested, I will let you know if it lives up to the high expectations. As one of my teachers in school once told me, "I'm known for being cynical, or even sarcastic -- myself, I prefer to call it healthy skepticism and mild irony."

Being a cynic isn't a lot of fun -- but for now, I would advise you to be at least healthily skeptical of what Attivio has to offer.