Phrase2VecGLM: Neural generalized language model–based semantic tagging for complex query reformulation in medical IR

Abstract

In fact-based information retrieval, stateof-the-art performance is traditionally achieved by knowledge graphs driven by knowledge bases, as they can represent facts about and capture relationships between entities very well. However, in domains such as medical information retrieval, where addressing specific information needs of complex queries may require understanding query intent by capturing novel associations between potentially latent concepts, these systems can fall short. In this work, we develop a novel, completely unsupervised, neural language model-based ranking approach for semantic tagging of documents, using the document to be tagged as a query into the model to retrieve candidate phrases from top-ranked related documents, thus associating every document with novel related concepts extracted from the text. For this we extend the word embeddingbased generalized language model (GLM) due to (Ganguly et al., 2015), to employ phrasal embeddings, and use the semantic tags thus obtained for downstream query expansion, both directly and in feedback loop settings. Our method, evaluated using the TREC 2016 clinical decision support challenge dataset, shows statistically significant improvement not only over various baselines that use standard MeSH terms and UMLS concepts for query expansion, but also over baselines using human expert-assigned concept tags for the queries, on top of a standard Okapi BM25-based document retrieval system.

Sequence-to-Set Semantic Tagging for Complex Query Reformulation and Automated Text Categorization in Biomedical IR using Self-Attention

“…We develop a novel sequence-to-set end-toend encoder-decoder-based neural framework for multi-label prediction, by training document representations using no external supervision labels, for pseudo-relevance feedback-based unsupervised semantic tagging of a large collection of documents. We find that in this unsupervised task setting of PRF-based semantic tagging for query expansion, a multi-term prediction training objective that jointly optimizes both prediction of the TFIDF-based document pseudo-labels and the log likelihood of the labels given the document encoding, surpasses previous methods such as Phrase2VecGLM (Das et al, 2018) that used neural generalized language models for the same. Our initial hypothesis that bidirectional or self-attentional models could learn the most efficient semantic representations of documents when coupled with a loss more effective than cross-entropy at reducing language model perplexity of document encodings, is corroborated in all experimental setups.…”

“…We ran several sets of experiments with various document encoders, employing pre-trained and fine-tuned embedding schemes like skip-gram (Mikolov et al, 2013a) and Probabilistic Fast-Text (Athiwaratkun et al, 2018), see Appendix B. The experimental setup used is the same as the Phrase2VecGLM (Das et al, 2018), the only other known system for this dataset, that performs "unsupervised semantic tagging of documents by PRF", for downstream query expansion. Thus we take this system as the current state-of-the-art system baseline, while our non-attention-based document encoding models constitute our standard baselines.…”

“…Similar to Das et al (2018), for the feedback loop based query expansion method, we had two separate human judgment-based baselines, one using the MeSH terms available from PMC for the top 15 documents returned in a first round of querying with Summary text, and the other based on human expert annotations of the 30 query topics, made available by the authors.…”

Sequence-to-Set Semantic Tagging: End-to-End Multi-label Prediction using Neural Attention for Complex Query Reformulation and Automated Text Categorization

“…In a related medical IR challenge (Roberts et al 2017) the authors specifically mention that with only six partially annotated queries for system development, it is likely that systems were either under-or over-tuned on these queries. Since the setup of the seq2set framework is an attempt to model the PRF based query expansion method of its closest related work (Das et al 2018) where the effort is also to train a neural generalized language model for unsupervised semantic tagging, we choose this system as the benchmark to compare against to our end-to-end approach for the same task.…”

“…Experiments -Unsupervised Task Setting We ran several sets of experiments with various document encoders, employing word embedding schemes like skipgram (Mikolov et al 2013) andProbabilistic Fasttext (Athiwaratkun, Wilson, andAnandkumar 2018). The experimental setup used is the same as the Phrase2VecGLM (Das et al 2018), the only other known system for this dataset, that performs "unsupervised semantic tagging of documents by PRF", for downstream query expansion. Thus we take this system as the current state-of-the-art system baseline, while our non-attention-based document encoding models constitute our standard baselines.…”