An experimental comparison of the effectiveness of computers and humans as search intermediaries

Abstract

An experimental computer intermediary system, CONIT, that assists users in accessing and searching heterogeneous retrieval systems has been enhanced with various search aids. Controlled experiments have been conducted to compare the effectiveness of the enhanced CONIT intermediary with that of human expert intermediary search specialists. Some 16 end users, none of whom had previously operated either CONIT or any of the four connected retrieval systems, performed searches on 20 different topics using CONIT with no assistance other than that provided by CONIT itself (except to recover from computer/software bugs). These same users also performed searches on the same topics with the help of human expert intermediaries who searched using the retrieval systems directly. Sometimes CONIT and sometimes the human expert were clearly superior in terms of such parameters as recall and search time. In general, however, users searching alone with CONIT achieved somewhat higher online recall at the expense of longer session times. We conclude that advanced experimental intermediary techniques are now capable of providing search assistance whose effectiveness at least approximates that of human intermediaries in some contexts. Also analyzed is the cost effectiveness of current intermediary systems. Finally, consideration is given to the prospects for much more advanced systems which would perform such functions as automatic data-base selection and the simulation of human experts, and thereby make information retrieval more effective for all classes of users.

Effects of search experience and subject knowledge on the search tactics of novice and experienced searchers

“…More sophisticated front-end systems like CITE, developed by Doszkocs (1983), was able to suggest keywords and medical subject headings based on user's initial queries but CITE is no longer used. Oddy's THOMAS program (1977) has experimented on assisting users in including synonyms; Meadow, Hewett, and Aversa (1982) have provided online search assistance to users of IIDA, OAKDEC (Meadow, 1988) offers diagnostic assistance to searchers too; Marcus (1983) has used a common command language in the CONIT interface to enable searchers to access multiple databases; and Zahir and Cheng (1992) recently completed an expert system prototype, ONLINE-EXPERT, for database selection. But these efforts have not yet been incorporated into commercial online systems.…”

Distributed Information Retrieval

“…Research on distributed IR can be traced back at least to Marcus, who in the early 1980's addressed resource description and selection in the EXPERT CONIT system, using expert system technology (Marcus, 1983). However, neither Marcus nor the rest of the research c o m m unity had access to a su ciently large experimental testbed with which to study the issues that became important during the 1990's: How to create solutions that would scale to large numbers of resources, distributed geographically, and managed by m a n y parties.…”

XLibris

“…While all of this information is available on the Web, the knowledge contained in Web pages is not necessarily put to use by users in their day-to-day activities. A recent study [11] suggested that users primarily access the Web through search engines. Users of search engines must first decide they need information, navigate to the appropriate engine, and then distill their request into keywords describing it.…”