A random walk on an ontology: Using thesaurus structure for automatic subject indexing

Abstract

Relationships between terms and features are an essential component of thesauri, ontologies, and a range of controlled vocabularies. In this article, we describe ways to identify important concepts in documents using the relationships in a thesaurus or other vocabulary structures. We introduce a methodology for the analysis and modeling of the indexing process based on a weighted random walk algorithm. . We also introduce a thesaurus-centric matching algorithm intended to improve the quality of candidate concepts. In all cases, the weighted random walk improves automatic indexing performance over matching alone with an increase in average precision (AP) of 9% for HEP, 11% for MeSH, 35% for NALT, and 37% for AGROVOC. The results of the analysis support our hypothesis that subject indexing is in part a browsing process, and that using the vocabulary and its structure in a thesaurus contributes to the indexing process. The amount that the vocabulary structure contributes was found to differ among the 4 thesauri, possibly due to the vocabulary used in the corresponding thesauri and the structural relationships between the terms. Each of the thesauri and the manual indexing associated with it is characterized using the methods developed here.

An automatic approach to weighted subject indexing—an empirical study in the biomedical domain

“…Another related research area is automatic subject indexing (Tzeras & Hartmann, 1993;Plaunt & Norgard, 1998;Ruch, 2006;Medelyan & Witten, 2008;Willis & Losee, 2013). Automatic subject indexing should be distinguished from automatic indexing (Willis & Losee, 2013).…”

“…Automatic subject indexing should be distinguished from automatic indexing (Willis & Losee, 2013). In automatic indexing (Salton, Wong, & Yang, 1975;Salton, Wu, & Yu, 1981), terms from the original items are generally used to represent the items.…”

“…Another related research area is automatic subject indexing (Medelyan & Witten, 2008;Plaunt & Norgard, 1998;Ruch, 2006;Tzeras & Hartmann, 1993;Willis & Losee, 2013). Automatic subject indexing should be distinguished from automatic indexing (Willis & Losee, 2013). In automatic indexing (Salton, Wong, & Yang, 1975;Salton, Wu, & Yu, 1981), terms from the original items are generally used to represent the items.…”

A semi-automatic indexing system based on embedded information in HTML documents

“…Some authors, Lancaster (2003) and Mai (1997) among them, analyze this procedure and the problems of identifying subjects. Others, such as Willis & Losee (2013) or Anderson (2001aAnderson ( , 2001b, review the most important aspects of manual and automatic subject indexing and also the differences between both systems.…”

The effect of assigning a metadata or indexing term on document ordering

“…This set of possible labels may be all terms in a natural language, or a limited set of terms from a thesaurus, producing a controlled vocabulary. Although machines may generate indexing representations that often represent a complex position in a machine‐developed term or topical space, most indexing systems use combinations of terms from a natural language, often consisting of a few terms (Foskett, ; Willis & Losee, in press). Metadata may be used, such as indexing, to represent topical aspects of documents, but metadata often expand beyond traditional indexing domains to representing data of all sorts as well as capturing nontopical aspects of data, such as authorship, security status, date of generation of the data, and so on (Greenberg, ).…”