An automatic approach to weighted subject indexing—an empirical study in the biomedical domain

Abstract

Subject indexing is an intellectually intensive process that has many inherent uncertaint ies.Existing manual subject indexing systems generally produce binary outcomes for whether or not to assign an indexing term. This does not sufficiently reflect the extent to which the indexing terms are associated with the documents. On the other hand, the idea of probabilistic or weighted indexing was proposed a long time ago and has seen success in capturing uncertainties in the automatic indexing process. One hurdle to overcome in implementing weighted indexing in manual subject indexing systems is the practical burden that could be added to the already intensive indexing process. This study proposes a method to infer automatically the associations between subject terms and documents through text mining. By uncovering the connections between MeSH descriptors and document text, we are able to derive the weights of MeSH descriptors manually assigned to documents. Our initial results suggest that the inference method is feasible and promising. The study has practical implications for improving subject indexing practice and providing better support for information retrieval.

Toward effective automated weighted subject indexing: A comparison of different approaches in different environments

“…Evaluating the performance of different weighting methods is not a trivial task. Our earlier studies (Lu & Mao, ; Lu et al, ) assessed the performance of different methods according to their ability to rank the major MeSH at the top. This evaluation method is cost‐effective.…”

“…The weighted mutual information between a subject descriptor h and an item i in document d is applied as in Lu and Mao (): where δ(i, h) is the weight of the pair <i, h>, which is obtained by: where tf i is the frequency of the item i in the document, N is the total number of documents in the corpus, df i & df h are the document frequencies (i.e., number of documents) of item i and subject descriptor h , respectively. The probabilities p(i,h) , p(i) , and p(h) are estimated by Maximum Likelihood Estimator (MLE) at the document level: …”

“…A more recent study by Zhang, Smith, Twidale, and Gao () revisited the need for a weighting mechanism for subject indexing. In Lu and Mao (), an automated approach that provides a concrete and cost‐effective implementation for weighted subject indexing has been proposed. Their initial experiments on a medical collection suggested the feasibility of the method.…”

Asiasana hallussa? – Opiskelijoiden tiedonhakukäyttäytyminen ja asiasanat osana sitä

“…Auktorisoimattomia luonnollisen kielen kuvailusanoja kutsutaan yleensä avainsanoiksi (keywords), ja niiden käyttö perustuu olettamukseen, että kirjoittaja ja käyttäjä käyttävät samaa käsitettä kuvaamaan samaa ilmiötä, eli tekstin ydinsanomaa (Taylor, 2004). Vaikka avainsanojen käyttö on yleistynyt, tarvitaan monien mielestä auktorisoituja asiasanoja luonnollisen kielen aiheuttamien ongelmien, kuten synonyymien, vuoksi (Gross, Taylor, & Joudrey, 2015;Lappalainen, Nykyri, & Palonen, 2013;Lu & Mao, 2015).…”