DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition

Abstract

This paper presents our pioneering effort for emotion recognition in conversation (ERC) with pre-trained language models. Unlike regular documents, conversational utterances appear alternately from different parties and are usually organized as hierarchical structures in previous work. Such structures are not conducive to the application of pre-trained language models such as XLNet. To address this issue, we propose an all-in-one XLNet model, namely DialogXL, with enhanced memory to store longer historical context and dialog-aware self-attention to deal with the multi-party structures. Specifically, we first modify the recurrence mechanism of XLNet from segment-level to utterance-level in order to better model the conversational data. Second, we introduce dialog-aware self-attention in replacement of the vanilla self-attention in XLNet to capture useful intra- and inter-speaker dependencies. Extensive experiments are conducted on four ERC benchmarks with mainstream models presented for comparison. The experimental results show that the proposed model outperforms the baselines on all the datasets. Several other experiments such as ablation study and error analysis are also conducted and the results confirm the role of the critical modules of DialogXL.

Cluster-Level Contrastive Learning for Emotion Recognition in Conversations

“…Considering the inter-speaker dependencies, DialogueRNN [29] proposes a global state RNN to model multi-party relations and emotional dynamics. Another branch of work leverages the strong context modelling ability of Transformer-based networks to model the dialogue as a whole [10], [30], [31]. To introduce more interpretable structures, there are also many works [32], [33], [34] that construct a graph on the dialogue, and devise graph neural networks to model ERC as a node-classification task.…”

“…DialogXL [10]: This PLM-based work uses dialog-aware self-attention to model inter-and intra-speaker dependencies, and utilises utterance recurrence to model long-range contexts.…”

“…Context modelling is a key challenge for ERC. The emotion of each utterance is influenced both by the previous utterances of the speaker and the responses of other participants [10]. Current methods mainly utilise Pre-trained Language Models (PLMs) [11] to deal with this challenge.…”

A Framework to Evaluate Fusion Methods for Multimodal Emotion Recognition

“…• Text modality: The model selected to process the text obtained from the audio transcription was based on the DialogXL, proposed in [57], which has obtained state of the art results when compared to other text processing models [58]. The output of the DialogXL was then passed to a neural network where each layer contained dialog-aware self-attention and utterance recurrence components.…”

Global and local feature fusion via long and short-term memory mechanism for dance emotion recognition in robot

“…Moreover, there is variability among speakers, their body language, and facial expressions (Kacur et al, 2021 ). Emotion recognition can greatly promote the integration and development of many different disciplines, such as graphics and image processing, artificial intelligence, human-computer interaction and psychology (Jiang and Yin, 2021 ; Shen et al, 2021 ). In human-computer interaction scenes with many different modes, the combination of emotion, posture, sound and other modes can make human-computer interaction experience more real.…”