Short Text Similarity with Word Embeddings

Abstract

Determining semantic similarity between texts is important in many tasks in information retrieval such as search, query suggestion, automatic summarization and image finding. Many approaches have been suggested, based on lexical matching, handcrafted patterns, syntactic parse trees, external sources of structured semantic knowledge and distributional semantics. However, lexical features, like string matching, do not capture semantic similarity beyond a trivial level. Furthermore, handcrafted patterns and external sources of structured semantic knowledge cannot be assumed to be available in all circumstances and for all domains. Lastly, approaches depending on parse trees are restricted to syntactically well-formed texts, typically of one sentence in length. We investigate whether determining short text similarity is possible using only semantic features-where by semantic we mean, pertaining to a representation of meaning-rather than relying on similarity in lexical or syntactic representations. We use word embeddings, vector representations of terms, computed from unlabelled data, that represent terms in a semantic space in which proximity of vectors can be interpreted as semantic similarity. We propose to go from word-level to text-level semantics by combining insights from methods based on external sources of semantic knowledge with word embeddings. A novel feature of our approach is that an arbitrary number of word embedding sets can be incorporated. We derive multiple types of meta-features from the comparison of the word vectors for short text pairs, and from the vector means of their respective word embeddings. The features representing labelled short text pairs are used to train a supervised learning algorithm. We use the trained model at testing time to predict the semantic similarity of new, unlabelled pairs of short texts. We show on a publicly available evaluation set commonly used for the task of semantic similarity that our method outperforms baseline methods that work under the same conditions.

Neural information retrieval: at the end of the early years

“…In this survey, we review neural models for textual similarity only if the model is evaluated for retrieval of similar textual units. Limiting to Similar Item Retrieval, we exclude works on neural models for general purpose textual similarity such as Hill et al (2016), Kenter and de Rijke (2015).…”

Neural Networks for Information Retrieval

“…First, using pre-trained word embeddings like combining traditional retrieval models with an embedding-based translation model [16,58], using pre-trained embeddings for query expansion to improve retrieval [57], and representing documents as Bag-of-Word-Embeddings (BoWE) [20,27]. Second, learning representations from scratch like learning representations of words and documents [28,32] and employing them in retrieval task [2,3], and learning representations in an end-to-end neural model for learning a speci c task like entity ranking for expert nding [53] or product search [52].…”

Data Quality Challenges in Twitter Content Analysis for Informing Policy Making in Health Care

“…Although research on natural language processing and text mining can be regarded as mature, it has largely focused on the assumption of well-written "long enough" documents [36]. Several methods for word and phrase similarity have been proposed in recent years [38,39], which aim at measuring similarity between text that may not contain any words in common. While not all these methods are directly applicable for social media, they represent promising approaches to identify related text content.…”