Self-attention based sentiment analysis with effective embedding techniques

Abstract

The problem of sparse vector representation exists in handling the large-scale data and also semantics of words are not considered in many existing works of sentiment analysis. Effective word embedding techniques improve the task of sentiment analysis by overcoming the above two problems. It is a challenging task, when the review is expressed in multiple sentences and the entire sentence needs to be considered instead of individual words to determine the sentiment. To achieve this task, we have proposed a novel LSTM-based deep learning architecture that applies a sentence embedding using universal sentence encoder along with an attention layer. To evaluate the proposed approach, we have carried out various experiments on IMDB data set. From the experimental results, it is observed that the proposed method is significantly better than the other word-embedding-based approaches with an improvement of 5% and we have achieved a F1 score of 89.12% for our approach.

SETAR: Stacking Ensemble Learning for Thai Sentiment Analysis Using RoBERTa and Hybrid Feature Representation

“…GRU is an extension of LSTM that is simpler and faster to train. Recently in SA, DL techniques that operate in conjunction with the convolution CNN, LSTM, BiLSTM, and GRU have been investigated to produce an even more accurate model, as can be seen from the work [25]- [27]. Word embedding, involving Word2Vec, fastText, and Glove, was successfully applied in sentiment classification studies, in order to capture sufficient sentiment information [27], [28].…”

English Machine Translation Model Based on an Improved Self-Attention Technology

“…In recent years, with the development of deep learning (DL), people have gradually begun to introduce deep learning to train a multilayer neural network to complete predetermined tasks [17]. In the field of natural language processing, such as English machine translation, question answering system, and reading comprehension, certain successes have been achieved [18]. e neural machine translation (NMT) system introduces deep learning technology; one of the mainstream technologies is to still retain the framework of statistical English machine translation, but to improve certain intermediate modules through deep learning technology, such as translation models, language models, and order adjustments [19].…”