The impact of MeSH (Medical Subject Headings) terms on information seeking effectiveness

Abstract

To what extent do MeSH (Medical Subject Headings) terms improve search effectiveness of different kinds of users? We observed four different kinds of searchers using an experimental information retrieval (IR) system: (1) search novices; (2) domain experts; (3) search experts and (4) medical librarians. The information needs were a subset of the relatively difficult topics originally created for the Text REtrieval Conference (TREC). By experimental design, we used 20 search topics in an IR user experiment to alleviate search topic variability. Effectiveness of retrieval was based on the relevance judgments set provided by TREC. Thirty-two participants searched either using a version of the system in which abstracts and MeSH terms were displayed or another version in which they had to formulate their own terms based only on the display of abstracts. We found that MeSH terms were more useful for domain experts than for search experts in terms of the precision measure, even though domain experts did not perceive that MeSH terms were useful. We speculate that because of the technical topics, only the domain experts had the knowledge to understand and therefore make use of the MeSH terms. The primary contributions of this research are: (1) assessment of relative impact of searchers characteristics of domain knowledge and search training on search effectiveness and (2) design and methodology for assessing the usefulness of controlled vocabulary. The effort to create MeSH terms is worthwhile for domain experts' searches on technical topics.

Still a Lot to Lose: The Role of Controlled Vocabulary in Keyword Searching

“…Liu reported, "experimental results strongly suggest that searchers with substantial domain knowledge can benefit from the use of MeSH terms in terms of the precision measure, even though their perception of the usefulness of MeSH terms did not agree with search performance." 45 Users' Difficulty with Subject Searching With so much evidence that scholars need more than keyword searching, why are some authors recommending that controlled vocabulary be abandoned? Several researchers have pointed out that many patrons cannot do subject searching successfully.…”

The Latin America and the Caribbean search strategy proposal

“…Finally, many scientists do not know how to use MEDLINE/PubMed to its full extent, especially when applying the MeSH terms (13). Liu demonstrated that MeSH terms differ in usefulness according to the user’s level of expertise (16). This means that the lack of familiarity with a specific topic and its MeSH term may lead to the loss of crucial information, even when using separate terms for each country, and could lead our brightening stars of LAC to fade away in the archive.…”

Interactive query formulation

“…Furthermore, the overall clarity scores of the users, including those of advanced searchers, were very low. Liu (2009, p. 73) reports a similar result, that the TREC information retrieval systems generally performed much better than human searchers, including domain experts and search experts, at least on hard biomedical searches.…”