Best bets: a worst practice?

  • 26-Jun-2008

As one of the authors of our Enterprise Search Report 2008, I've been spending a lot of time lately looking at search technology and talking to folks who care deeply about the subject (i.e., vendors and their customers).

One thing everyone seems to agree on is that providing relevant results to the user is a very hard problem indeed. People don't want to enter a few keywords and then get 10,000 hits on documents that contain those keywords, 9,999 of which might be irrelevant. They want pointers to the one or two documents that are relevant.

The signal-to-noise problem is so thorny that many enterprise search products include an optional feature known as "best bets." The idea is that certain very common searches should point to particular documents (or intranet pages) that are known, or presumed, to apply. Imagine that a lawyer working for a large legal firm logs into the company portal and searches on "poison pill." A thousand hits might come back, of which 990 are related to medications, allergic reactions, toxicity, malpractice, and so on, even though all the person was really looking for was a link to the company's "merger and acquisitions" resource page. ("Poison pill" is a term for tactics a company can use to fend off hostile takeover attempts.) The idea of "best bets" is that you rig the system to promote the company's "M&A resources" link to the top of the hit list whenever someone does a search on "poison pill."

Sometimes "best bets" refers to presenting the user with a recommendation when, say, several repositories exist, one or more of which could be better-suited to a given search than the others. ("Would you like to search the Parts Catalog for this?") This is more of a navigational scenario. That's not really what I'm talking about here. I'm talking about  the practice of biasing search results by hard-coding certain answers to certain common queries.

Setting up "best bets" is typically a manual process. A person in IT will use search analytics to determine the most common search queries and the most-followed links associated with them. Then those associations will be captured in a database and wired into the search software in such a way that when a user issues a query for which a best bet already exists, the best-bet link(s) will automatically be shown at the top of the results page (either as a regular hit or under a separate heading of "Best Bets").

Not everyone thinks the "best bets" mechanism is a good idea. The problem is that, fundamentally, it's a hack. It's arguably the worst kind of hack in that it involves serious amounts of human intervention. Someone has to create the best-bet database. (Typically there will be hundreds, if not thousands, of best-bet links.) Then the database has to be updated and kept fresh as user needs change and documents are added to or dropped from the system.

In point of fact, the search software should do all this for you. After all, that's its job: to return relevant results (automatically) in response to queries. Why would you sink tens (or hundreds) of thousands of dollars into an enterprise search system only to override it with a manually assembled collection of point-hacks?

Sure, search is a hard problem. But if your search system is so poor at delivering relevant results that it can't figure out what your users need without someone in IT explicitly telling it the answer, maybe you should search for a new search vendor. (And for help with that, see Enterprise Search Report 2008, a free sample of which is available right here.)

Our customers say...

"As high-end web consultants, we use a variety of research tools to keep on top of the evolving technology platforms. Among these, Real Story Group is by far the best--a sort of bible for our CMS consulting practice. We've used The Web CMS Research both to get a jump on understanding products in advance of a formal CMS selection, as well as to start the conversation about potential partnerships. It's also great when a client asks you about an obscure product -- and you can get back to them within an hour with a grounded, thoughtful appraisal of the product, knowing that the information is accurate and written by analysts with real perspective."

John Berndt, President and CEO, The Berndt Group, Ltd.

Other posts

DAM Scenario 13: Media Editing & Assembly

  • February 22, 2019

This scenario overlaps a bit with Media Asset Management (MAM) technology and desktop-based editing suites. DAM vendors have been encroaching on both domains in recent years....

How Did Our 2018 Predictions Fare?

  • February 18, 2019

By my count we were 8.5 out of 12, or once again about 71%. That's pretty typical for the 15 years we've been prognosticating......

Non-disclosure is a non-starter

  • February 15, 2019

It bears discussing one more time: vendor NDAs are not conducive to telling the real story to enterprise technology customers...

DAM Scenario 12: Media Library Review and Approval

  • February 15, 2019

This scenario occurs regularly in marketing departments and traditional entertainment and media companies. Colleges and universities use it to provide rich media material or for e-learning modules....