Learning Semantic Similarity for Very Short Texts

Abstract

Levering data on social media, such as Twitter and Facebook, requires information retrieval algorithms to become able to relate very short text fragments to each other. Traditional text similarity methods such as tf-idf cosine-similarity, based on word overlap, mostly fail to produce good results in this case, since word overlap is little or non-existent. Recently, distributed word representations, or word embeddings, have been shown to successfully allow words to match on the semantic level. In order to pair short text fragments - as a concatenation of separate words - an adequate distributed sentence representation is needed, in existing literature often obtained by naively combining the individual word representations. We therefore investigated several text representations as a combination of word embeddings in the context of semantic pair matching. This paper investigates the effectiveness of several such naive techniques, as well as traditional tf-idf similarity, for fragments of different lengths. Our main contribution is a first step towards a hybrid method that combines the strength of dense distributed representations - as opposed to sparse term matching - with the strength of tf-idf based methods to automatically reduce the impact of less informative terms. Our new approach outperforms the existing techniques in a toy experimental set-up, leading to the conclusion that the combination of word embeddings and tf-idf information might lead to a better model for semantic content within very short text fragments.Comment: 6 pages, 5 figures, 3 tables, ReLSD workshop at ICDM 1

Learning semantic sentence representations from visually grounded language without lexical knowledge

“…So far, existing sentence embedding methods often require (pretrained) word embeddings [10,12], large amounts of data [8], or both [13,11]. While word embeddings are successful at enhancing sentence embeddings, they are not very plausible as a model of human language learning.…”

Time-aware domain-based social influence prediction

“…The work by Wang et al, [65] proposed a social media analytics engine that employs a fuzzy similarity-based classification method to automatically classify text messages into sentiment categories (positive, negative, neutral and mixed), with the ability to identify their prevailing emotion categories (e.g., satisfaction, happiness, excitement, anger, sadness, and anxiety). Others attempted to identify the semantic similarity of very short texts in Twitter and Facebook [66]. Also, a lexical similarity-based approach for extracting subjectivity in documents extracted from social media was proposed in [67].…”

Corpus Construction and Semantic Analysis of Indonesian Image Description

“…The word-embedding method has successfully identified the semantic distances between two sentences better than the traditional approach for text similarity (e.g., the distance of the tf-idf vector) [18]. In this research, we used two wordembedding methods to calculate the semantic distance between Eng2Ind Translation and Ind Caption:…”