Computational Analyses of Thin-Sliced Behavior Segments in Session-Level Affect Perception

Lin, Wei-Cheng; Lee, Chi-Chun

doi:10.1109/taffc.2018.2816654

Cited by 4 publications

(1 citation statement)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compute the gradient log-likelihood function, i.e., Fisher scoring (indicating the direction of λ to better fitx l ), with respect to the first and second order statistics of the learned latent VaDE-GMM parameters to further encode a sequence of acoustic latent representationx l into a fixed-length representation (also terms as GMMbased Fisher-vector encoding [26]). The use of Fisher-vector encoding has been shown to be competitive in speech-related tasks of paralinguistic recognition [27], presentation scoring [28], and emotion recognition [29,30]. The dialog-level acoustic vectors that integrates both the general representation and the dyad-specific dynamics is derived by concatenating the general Fisher-scoring vector with the dyad-specific Fisher-scoring vector.…”

Section: Dialog-level Emotion Recognitionmentioning

confidence: 99%

Encoding Individual Acoustic Features Using Dyad-Augmented Deep Variational Representations for Dialog-level Emotion Recognition

Lee

2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

Face-to-face dyadic spoken dialog is a fundamental unit of human interaction. Despite numerous empirical evidences in demonstrating interlocutor's behavior dependency in dyadic interactions, few technical works exist in leveraging the unique pattern of dynamics in task of advancing emotion recognition during face-to-face settings. In this work, we propose a framework of encoding an individual's acoustic features with dyadaugmented deep networks. The dyad-augmented deep networks includes a general variational deep Gaussian Mixture embedding network and a dyad-specific fine-tuned network. Our framework utilizes the augmented dyad-specific feature space to incorporate the unique behavior pattern emerged when two people interact. We perform dialog-level emotion regression tasks in both the CreativeIT and the NNIME databases. We obtain affect regression accuracy of 0.544 and 0.387 for activation and valence in the CreativeIT database (a relative improvement of 4.41% and 4.03% compared to using features without augmenting the dyad-specific representation), and we obtain 0.700 and 0.604 (4.48% and 4.14% relative improvement) for regressing activation and valence in the NNIME database.

show abstract

Section: Dialog-level Emotion Recognitionmentioning

confidence: 99%

Encoding Individual Acoustic Features Using Dyad-Augmented Deep Variational Representations for Dialog-level Emotion Recognition

Lee

2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Ramaswamy,

Palaniswamy

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Automatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices.This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Cognitive Computing Technologies > Artificial Intelligence

show abstract

Use of affect context in dyadic interactions for continuous emotion recognition

Fatima

Erzin

2021

Speech Communication

View full text Add to dashboard Cite

Computational Analyses of Thin-Sliced Behavior Segments in Session-Level Affect Perception

Cited by 4 publications

References 53 publications

Encoding Individual Acoustic Features Using Dyad-Augmented Deep Variational Representations for Dialog-level Emotion Recognition

Encoding Individual Acoustic Features Using Dyad-Augmented Deep Variational Representations for Dialog-level Emotion Recognition

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Use of affect context in dyadic interactions for continuous emotion recognition

Contact Info

Product

Resources

About