Cross-Domain Sentiment Encoding through Stochastic Word Embedding

Abstract

Sentiment analysis is an important topic concerning identification of feelings, attitudes, emotions and opinions from text. To automate such analysis, a large amount of example text needs to be manually annotated for model training. This is laborious and expensive, but the cross-domain technique is a key solution to reducing the cost by reusing annotated reviews across domains. However, its success largely relies on the learning of a robust common representation space across domains. In the recent years, significant effort has been invested to improve the cross-domain representation learning by designing increasingly more complex and elaborate model inputs and architectures. We support that it is not necessary to increase design complexity as this inevitably consumes more time in model training. Instead, we propose to explore the word polarity and occurrence information through a simple mapping and encode such information more accurately whilst managing lower computational costs. The proposed approach is unique and takes advantage of the stochastic embedding technique to tackle cross-domain sentiment alignment. Its effectiveness is benchmarked with over ten data tasks constructed from two review corpora and it is compared against ten classical and state-of-the-art methods.

Refined Global Word Embeddings Based on Sentiment Concept for Sentiment Analysis

“…With the development of NLP, Sentiment Analysis has been paid more attention by researchers and many efforts have been made in word embeddings. Jiang et al [5] proposed Bag-of-words text representation method based on sentiment topic words, which is composed of deep neural network, sentiment topic words and context information, and performed well in Sentiment Analysis; Rezaeinia et al [6] proposed refined word embeddings method based on Part-of-Speech(POS) tagging technology and sentiment lexicons, which improved the performance of pre-trained word embeddings in Sentiment Analysis; Pham et al [7] proposed a joint model of multiple Convolutional Neural Networks (CNNs), which is focused on word embeddings from Word2Vec, GloVe and the one-hot character vectors, and it achieved good performance in aspect sentiment classification tasks; Zhou et al [8] constructed a text representation model containing TF-IDF and topic features based on LDA for Sentiment Analysis, which reduced the dimension of word vector space in the traditional representation model; Han et al [9] built a hybrid neural network model using convolutional neural networks and long short-term memory(LSTM) for document representation, and it incorporated user's and product's information; Devlin J. et al [10] proposed the BERT model to represent text, which can better reflect the modifying relationship between words in texts, and it had good performance in Sentiment Analysis tasks; Liu et al [11] proposed latent topic information of the text that used Neural topic model into word-level semantics representations to deal with the problem of data sparsity, and presented a new topic-word attention mechanism to explore the semantics of words from the perspective of topic word association; Li et al [12] proposed a framework that combined different levels of prior knowledge into word embeddings for Sentiment Analysis, which improved the performance of Sentiment Analysis; Xu et al [13] proposed an improved word representation method, which integrated the contribution of sentiment information into the traditional TF-IDF algorithm and generated weighted word vectors, and the method had higher F1 score; Peters M. E. et al [14] proposed a text representation model based on deep learning framework, and it constructed an English text representation model which contained grammar feature, semantics feature and sentiment feature by training a large number of sentiment text corpus; Hao et al [15] proposed a method for cross domain sentiment classification using random embeddings, which retained similar structure in embedding space and achieved well results in the task of Sentiment Analysis; Usama et al [16] merged multilevel features which are from different layers of the same network and different network architectures to improve the accuracy of Sentiment Analysis; Majumder et al [17] demonstrated the correlation between sarcasm detection and sentiment classification, and proposed a multitasking learning framework to improve the performance of two tasks; Ma et al [18] proposed Sentic...…”

Latent Dirichlet Allocation and t-Distributed Stochastic Neighbor Embedding Enhance Scientific Reading Comprehension of Articles Related to Enterprise Architecture

“…t-SNE, similar to transformers, examines the proximity of words to each other [27]. Hao et al tackle cross-domain sentiment alignment by applying stochastic word embedding [28].…”

An Effective BERT-Based Pipeline for Twitter Sentiment Analysis: A Case Study in Italian

“…Furthermore, Yu et al [ 34 ] presented a new way to refine word embeddings for sentiment analysis using intensity scores from sentiment lexicons. Moreover, Hao et al [ 35 ] applied a novel stochastic embedding technique for cross-domain sentiment classification, preserving the similarity in the embedding space. Finally, Ali et al [ 36 ] proposed a system that retrieved transport content from social networks, representing the documents with word embedding techniques and achieving an effective approach to sentiment classification with 93% accuracy.…”