DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

Abstract

Emotion recognition in conversation (ERC) has received much attention, lately, from researchers due to its potential widespread applications in diverse areas, such as health-care, education, and human resources. In this paper, we present Dialogue Graph Convolutional Network (DialogueGCN), a graph neural network based approach to ERC. We leverage self and inter-speaker dependency of the interlocutors to model conversational context for emotion recognition. Through the graph network, DialogueGCN addresses context propagation issues present in the current RNN-based methods. We empirically show that this method alleviates such issues, while outperforming the current state of the art on a number of benchmark emotion classification datasets. * Corresponding author My head is aching Frustrated

Multi-Modality Emotion Recognition Model with GAT-Based Multi-Head Inter-Modality Attention

“…Each video contains a single dyadic dialogue that is segmented into utterances and annotated with emotion labels. We strictly follow Majumder et al [ 23 ] and Ghosal et al [ 27 ] and split the datasets into training and testing sets with a rough 80/20 ratio: 5810 training samples and 1623 testing samples.…”

“…To utilize contextual information in dialogues for detecting emotions, Hazarika et al [ 44 ] employed two distinct gate recurrent networks (GRUs) for different speakers and fed two GRUs with the utterance context from the dialogue. Majumder et al [ 23 ] and Ghosal et al [ 27 ] constructed a model that separately analyzed the global emotion states and party states from dialogue contexts and calculated the effects of the interlocutors and individual historical utterances to detect emotions. However, in practical implementations, since agents hard to express emotions affect users, an effective model is required that can focus on the mental state trends of only the users in the dialogue.…”

“…Graph neural networks (GNNs) have emerged to deal with such information. At the same time, studies discovered that GNN can also promote the performance of emotion recognition [ 27 , 52 ]. Zhang et al [ 52 ] recognized emotions with EEG-channel signals by a graph convolutional broad network (GCB-Net), which explored the deeper-level information of structured data.…”

“…Zhang et al [ 52 ] recognized emotions with EEG-channel signals by a graph convolutional broad network (GCB-Net), which explored the deeper-level information of structured data. Ghosal et al [ 27 ] adopted a graph convolutional network (GCN) to deeply capture the inter-relationship utterances/emotions in a dialogue. In our study, we regarded the hidden features from different modalities as a node and employed a graph attention network (GAT) to emphasize the strengths of each modality.…”

“…We also propose a graph attention network (GAT) for constructing a decision-level fusion strategy. We evaluated our proposed method on IEMOCAP [ 20 ] and compared the evaluation results of two baselines, DialogueRNN [ 23 ] and DialogueGCN [ 27 ], which are the most recent studies that report state-of-the-art results. The following are the contributions of this study: We propose a strategy for augmenting audio samples based on the entropy weight method.…”