A deep network model for paraphrase detection in short text messages

Abstract

This paper is concerned with paraphrase detection. The ability to detect similar sentences written in natural language is crucial for several applications, such as text mining, text summarization, plagiarism detection, authorship authentication and question answering. Given two sentences, the objective is to detect whether they are semantically identical. An important insight from this work is that existing paraphrase systems perform well when applied on clean texts, but they do not necessarily deliver good performance against noisy texts. Challenges with paraphrase detection on user generated short texts, such as Twitter, include language irregularity and noise. To cope with these challenges, we propose a novel deep neural network-based approach that relies on coarse-grained sentence modeling using a convolutional neural network and a long short-term memory model, combined with a specific fine-grained word-level similarity matching model. Our experimental results show that the proposed approach outperforms existing state-of-the-art approaches on user-generated noisy social media data, such as Twitter texts, and achieves highly competitive performance on a cleaner corpus.

A multi-cascaded model with data augmentation for enhanced paraphrase detection in short texts

“…4 Ferreira et al (2018) [44] 74.08 83. 1 Agarwal et al (2018) [4] 77.7 84.5 Arora and Kansal (2019) [45] 79.0 − Our model 78.3 84.8…”

“…For instance, the AskUbuntu dataset [27] contains very few annotations, thus limiting the generalization performance of the model [13]. The ability to augment the data with additional sound annotations without requiring human intervention can improve the performance of deep models [4,13]. Such data augmentation has been shown to be fruitful for data analytics when only a piece of limited ordinal information about the pairwise distance between objects is provided [28,29,30].…”

“…We employ a set of NLP/linguistic features in our experiments as it has been shown that including linguistic features for paraphrase identification in short text can improve the performance of deep learning models [4]. We identify the following linguistic and statistical features to be used alongside learned features in our multi-cascaded model.…”

“…crowd-sourcing) is costly [16]. Therefore, [4] and [13] add to the training set each labeled pair also in the reversed order. However, this simple data augmentation strategy can be extended in a systematic manner by relying upon set and graph theory.…”

Deep Contextualized Pairwise Semantic Similarity for Arabic Language Questions

“…The model computes the average of vectors of all words and n-grams in a sentence. Several studies have shown successful results produced by Sent2vec [32,33]. The model learns a context embedding v w and target embedding u w for each word w in the vocabulary, with h number of embedding dimensions.…”