Net Q & A

Question of the Month: April 2000

What is the hottest trend in Internet research? 

Answer

The use of human intelligence to supplement computers and make searching easier.

Direct Use of Human Intelligence

In the wonderful recent book Law of the Super Searchers, the expert legal researchers interviewed said they preferred to start most searches using not search engines, but directories, collections of links that had been assembled by human beings, preferably experts on the subject to be searched.  

A key example of this is the Open Directory Project, a sort of "open source" Internet directory.  It uses scores of human volunteers to compile a mammoth, high quality directory to the Internet.  The approach gives it the potential to be more complete and timely than perennial favorite directories like Yahoo.

Indirect Use of Human Intelligence

The most difficult part about using search engines is sorting out the results you want from the great mass of sites that are useless to you.  Two exciting new search engines make indirect use of human intelligence to ease this daunting task.

Direct Hit is a deservedly acclaimed new search engine that owes much of its success to the clever way in which it incorporates human intelligence to refine its results. When ranking search engine results, it places heavy weight on the popularity of a page (i.e., how often a page is visited by other human beings). The result is a sort of "collaborative filtering," with the judgments of others affecting the rankings. Their slogan is: "One Search Engine, Millions of Minds."

Google is an even more popular search engine that takes a more refined approach. It gives heavy weight in ranking sites to which sites other sites have linked to. The more web sites that link to a particular site, the higher it ranks. Google also gives more credit to links from sites it considers significant (i.e., those that have many incoming links themselves, a sort of "rate the rater" approach.

Google's approach leads to results that are known as "high precision, low recall." This means that while every obscure site in the world that mentions your search terms may not be included, the results you seek are more likely to be among the first few returned. Since most searches don't need to be exhaustive, Google's recall/precision tradeoff is just fine for most research projects, so the search engine has won nearly universal praise.

Jerry Lawson

This page last revised: January 01, 2002.