Transformer-Based Graph Convolutional Network for Sentiment Analysis

Abstract

Sentiment Analysis is an essential research topic in the field of natural language processing (NLP) and has attracted the attention of many researchers in the last few years. Recently, deep neural network (DNN) models have been used for sentiment analysis tasks, achieving promising results. Although these models can analyze sequences of arbitrary length, utilizing them in the feature extraction layer of a DNN increases the dimensionality of the feature space. More recently, graph neural networks (GNNs) have achieved a promising performance in different NLP tasks. However, previous models cannot be transferred to a large corpus and neglect the heterogeneity of textual graphs. To overcome these difficulties, we propose a new Transformer-based graph convolutional network for heterogeneous graphs called Sentiment Transformer Graph Convolutional Network (ST-GCN). To the best of our knowledge, this is the first study to model the sentiment corpus as a heterogeneous graph and learn document and word embeddings using the proposed sentiment graph transformer neural network. In addition, our model offers an easy mechanism to fuse node positional information for graph datasets using Laplacian eigenvectors. Extensive experiments on four standard datasets show that our model outperforms the existing state-of-the-art models.

Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network

“…It is commonly appreciated that accuracy is a standard metric used to evaluate the overall sentiment analysis performance [38][39][40][41][42]. According to the research by Pei et al [41], Rao et al [16], and Behera et al [43], this paper adds two evaluation parameters (F1-Measureand MSE) to evaluate the performance of sentiment analysis.…”

An Easy Partition Approach for Joint Entity and Relation Extraction

“…To obtain improved task-specific features, the authors of [15] proposed a neuron-level partition filter network that learns three feature representations at the neuron level, and the structure has been proven effective in multiple tasks, but the method is a bit complex. The pretraining language model (PLM) has achieved state-of-the-art performance in various NLP tasks, including machine translation [16,17] and sentiment analysis [18]. To leverage the latent contextual space of PLM, we propose a simple but effective approach, called easy partition approach for relation extraction (EPRE).…”