Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN

Abstract

Convolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excessive trainable parameters, and long training time. The classification effect of traditional model with cross entropy loss as loss function is undesirable since sample category imbalance as well as ease and difficulty of sample classification is not taken into account. In order to solve these problems, the model BiGRU-Att-HCNN is proposed on the basis of bidirectional gated recurrent unit (BiGRU), attention, and hybrid convolutional neural networks. In this model, BiGRU and self-attention are combined to acquire global information, and key information weight is supplemented. Two parallel convolutions (dilated convolution and standard convolution) are used to obtain multi-scale characteristic information with relatively less parameters, and the standard convolution is replaced with depthwise separable convolution with two-step calculations. Traditional max-pooling and average-pooling are discarded, and global average pooling is applied to substitute the pooling layer and the fully-connected layer simultaneously, making it possible to substantially decrease the number of model parameters and reduce over-fitting. In our model, focal loss is used as the loss function to tackle the problems of unbalanced sample categories and hard samples. Experimental results illustrate that in terms of multiple indicators, our model outperforms the 15 benchmark models, even with intermediate number of trainable parameters.

An Experimental Analysis of Deep Neural Network Based Classifiers for Sentiment Analysis Task

“…They experimented with both randomized and pre-trained word vectors for generating word vectors in embedding layer. Zhu et al [30] used a similar architecture based on attention mechanism word embeddings which were passed through a BiGRU layer for capturing long-distance contextual semantics. Self-attention is then applied on BiGRU layer's output to compute word similarity across sentence subsequently emphasizing word with strong emotions.…”

A multi-scale sarcasm sentiment recognition algorithm incorporating sentence hierarchical representation

“…(6) BiGRU-Att-HCNN [19]: using an attention mechanism to fuse the information extracted by BiGRU and HCNN models to enrich the semantic information and feature information of the sentence.…”

A Multi-Dimensional Text Sentiment Analysis Method Based on Joint Network

“…When the convolution kernel window size is set to 5, the prediction accuracy is the highest. Figure 2(b) shows the prediction accuracy when using different combinations of convolution kernels of different sizes, and it can be seen that when the combination of convolution kernel window sizes is (3,4,5), the capsule network obtains higher prediction accuracy.…”

“…Yang et al [3] combined the BERT model with the capsule network and proposed an enhanced capsule network to accurately feedback the real word-of-mouth of the movie through the user's comments in the sentiment analysis based on social media comments. The BiGRU model is composed of two GRU models in opposite directions superimposed up and down [4], and the single GRU model can only obtain the one-way above or below information of the text [5]. In this paper, BiGRU is used instead of GRU to better capture the bidirectional semantic dependence of the text.…”