Online Attention for Interpretable Conflict Estimation in Political Debates

Abstract

Conflict arises naturally in dyadic interactions when involved individuals act on incompatible goals, interests, or actions. In this paper, the problem of conflict intensity estimation from audiovisual recordings is addressed. To this end, we propose an online attention-based neural network in order to learn a mapping from a sequence of audiovisual features to time-series describing conflict intensity. The proposed method is evaluated by conducting experiments in conflict intensity estimation by employing the CONFER dataset. Experimental results indicate the superiority of the proposed model compared to the state of the art. Furthermore, we demonstrate that by incorporating sparsity in the model, the origin of conflict can be traced back to specific key frames facilitating the interpretation of conflict escalation.

ConflictNET: End-to-End Learning for Speech-Based Conflict Intensity Estimation

“…Recently, a multi-modal conflict estimation method used a concatenation of audio and visual features as input to an LSTM-based encoder-decoder architecture with attention. This method focuses on visual features (facial gestures) and uses 65 audio Low-Level Descriptors (LLD) features, sampled at 25 Hz [23]. While hand-crafted features may facilitate interpretation of specific characteristics of the speech signal that are used as predictors for the task at hand, we aim to explore if an end-to-end learning framework can be used for a complex paralinguistic task such as verbal conflict intensity estimation by automatically learning relevant acoustic features for this task.…”

“…speech overlap ratio using BLSTM DNN [9] IS13 & over. forward-backward pass SVR [3] IS10 & IS13 overlap detection using SVR SVM + backward selection [23] FPF & LLD LSTM based encoder-decoder network [20] raw speech End-to-End Convolutional Neural Network Ours raw speech End-to-End CRNN with attention…”