Are ants really better than spiders?

Enterprise search vendor Autonomy is touting its "ant" technology as an alternative to traditional spidering technologies, but regrettably does not really explain what it does. Ant technology means algorithms that crawl over sparse data. The core concept is influenced by the discipline of mereology. Algorithms work over data like a colony of working ants, feeding revised weights into subsequent calculations to add tiny bits of incrementally "better" data. So, when ant technology hits a table of data about people with diabetes, the algorithm can generate a list of people without diabetes, people with diabetes, and perhaps even people who are tending toward or away from diabetes. The same idea applies to textual information. Regardless of the corpus, the ant algorithms swarm over the text, generating values and incrementally better values over time. The technology is neither new nor universally useful; it is most germane to signal analysis, drug analysis, and practical problems where there is no one best answer. Like other advanced search methods this is not cheap: computationally you can see that CPU capacity, fast disk I/O, and abundant memory are needed to make recursive algorithms work. Text applications require NSA-level computer power and quite a bit of fiddling to get the system to work.


Our customers say...

"I've seen a lot of basic vendor comparison guides, but none of them come close to the technical depth, real-life experience, and hard-hitting critiques that I found in the Search & Information Access Research. When I need the real scoop about vendors, I always turn to the Real Story Group."


Alexander T. Deligtisch, Co-founder & Vice President, Spliteye Multimedia
Spliteye Multimedia

Other Posts