Bridging the gap: Incorporating a semantic similarity measure for effectively mapping PubMed queries to documents

Abstract

The main approach of traditional information retrieval (IR) is to examine how many words from a query appear in a document. A drawback of this approach, however, is that it may fail to detect relevant documents where no or only few words from a query are found. The semantic analysis methods such as LSA (latent semantic analysis) and LDA (latent Dirichlet allocation) have been proposed to address the issue, but their performance is not superior compared to common IR approaches. Here we present a query-document similarity measure motivated by the Word Mover’s Distance. Unlike other similarity measures, the proposed method relies on neural word embeddings to compute the distance between words. This process helps identify related words when no direct matches are found between a query and a document. Our method is efficient and straightforward to implement. The experimental results on TREC Genomics data show that our approach outperforms the BM25 ranking function by an average of 12% in mean average precision. Furthermore, for a real-world dataset collected from the PubMed® search logs, we combine the semantic measure with BM25 using a learning to rank method, which leads to improved ranking scores by up to 25%. This experiment demonstrates that the proposed approach and BM25 nicely complement each other and together produce superior performance.

Neural information retrieval: at the end of the early years

“…This approach for learning bilingual word embeddings is referred to as Bilingual word Embeddings Skip-Gram (BWESG). Azimi et al (2015), Zhai et al (2016a, b) 8.2 Learn Learn to predict Grbovic et al (2015a, b), Zhang et al (2016a) 9 Similar item retrieval 9.1.1 Related document search Aggregate Explicit Kim et al (2016), Kusner et al (2014) 9.1.2 Learn Learn to autoencode Salakhutdinov and Hinton (2009) Learn to predict Dai et al (2014), Djuric et al (2015), Le and Mikolov (2014) 9.2 Detecting text reuse Aggregate Explicit Zhang et al (2014) 9.3.1 Similar Question Retrieval…”

“…Their model outperforms these baselines. Kim et al (2016) propose another version of WMD specific to query-document similarity. The high computational cost of WMD is tackled by mapping queries to documents using a word embedding model trained on a document set.…”

Toward Automated Content Feedback Generation for Non-native Spontaneous Speech

“…• Query-document Similarity (QD): Responses are generally much longer than Key Points and WM-distance may assign unfairly low similarity scores to responses with extra information. To address this issue, we use metrics designed for information retrieval (Kim et al, 2016). For each word in the Key Point, the algorithm finds the word with the maximum similarity from a response n-gram, where the similarity score is the cosine similarity between two corresponding word embeddings.…”

Semantically-informed distance and similarity measures for paraphrase plagiarism identification

“…Although these methods have been widely applied for measuring the degree of paraphrases between two given texts, just [17] evaluates its relevance for plagiarism detection. More recently, [5,12] discussed the use of semantic information without depending on any external knowledge resource. Particularly, they proposed using distributive representations, such as word2vec [15], in the task of plagiarism detection.…”