TGCN-Bert Emoji Prediction in Information Systems Using TCN and GCN Fusing Features Based on BERT

Abstract

In recent studies, graph convolutional neural networks (GCNs) have been used to solve different natural language processing (NLP) tasks. However, few researches apply graph convolutional networks to short text classification. Emoji prediction, as a complex sentiment analysis task, has received even less attention. In this work, the authors propose TGCN-Bert which combines pre-trained BERT temporal convolutional networks (TCNs) and graph convolutional networks for short text classification and emoji prediction. They initialize the nodes with the help of BERT and define the edges in text graph based on the term frequency-inverse document frequency (TF-IDF) and positive point-wise mutual information (PPMI). They employ the model for emoji prediction task, and a metric based on emoji clustering is developed to better measure the validity of emoji prediction results. To validate the performance of TGCN-Bert, they compare it with other GCN variants on short text classification datasets and emoji prediction datasets; experiments show that TGCN-Bert achieves better performance.

Deep Learning-Based Sentiment and Stance Analysis of Tweets About Vaccination

“…BERT models have been trained and described for many languages so far, including Spanish (Cañete et al, 2020), French (Le et al, 2019, Italian (Polignano et al, 2019), and Chinese (Cui et al, 2021). BERT models have also been used to solve various NLP tasks, as reported in the literature (Runmei et al, 2022;Yang et al, 2023).…”