Query Reformulation Based on Word Embeddings: A Comparative Study

Abstract

Formulating effective queries for retrieving domain-specific content from the Web and social media is very important for practitioners in several fields, including law enforcement analysts involved in terrorism-related investigations. Query reformulation aims at transforming the original query in such a way, so as to increase the search effectiveness by addressing the vocabulary mismatch problem. This work presents a study comparing the performance of global versus local word embeddings models when applied for query expansion. Two query expansions methods are employed (i.e., CombSum and Centroid) for defining the most similar terms to each query term, based on Glove pre-trained global embeddings and local models trained on four large-scale benchmark and one terrorism-related datasets. We assessed the performance of the global and local models on the benchmark datasets based on commonly used evaluation metrics, and performed a qualitative evaluation of the respective models on the terrorism-related dataset. Our findings indicate that the local models yield promising results on all datasets.

Query Reformulation for Indonesian Question Answering System Using Word Embedding of Word2Vec

“…A new query is generated based on calculating the similarity of meaning between words based on the context of a corpus. The method used is word2vec which is based on an artificial conditional network model called Continuous Bag-of-Words (CBOW) [14]. The word2vec training corpus from Wikipedia Dump is about 600MB in size.…”