ABAW: Learning from Synthetic Data &amp; Multi-task Learning Challenges

Kollias, Dimitrios

doi:10.1007/978-3-031-25075-0_12

Cited by 35 publications

(25 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We report results by CCC and F 1 score for the three subchallenges in table 1 on the validation set. For the facial expression classfication, the authors for the baseline [10] perform the pre-trained VGG16 network on the VGGFACE dataset and get softmax probabilities for the 8 expression predictions. In our proposed model, there are virious effective data augmentation strategies employed to alleviate the problems of sample imbalance during model training.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Spatial-temporal Transformer for Affective Behavior Analysis

Zou¹,

Wang²,

Wen³

et al. 2023

Preprint

View full text Add to dashboard Cite

The in-the-wild affective behavior analysis has been an important study. In this paper, we submit our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes V-A Estimation, Facial Expression Classification and AU Detection Sub-challenges. We propose a Transformer Encoder with Multi-Head Attention framework to learn the distribution of both the spatial and temporal features. Besides, there are virious effective data augmentation strategies employed to alleviate the problems of sample imbalance during model training. The results fully demonstrate the effectiveness of our proposed model based on the Aff-Wild2 dataset.

show abstract

Section: Resultsmentioning

confidence: 99%

“…al. [10][11][12][13][14][15][16][17][18][19]28] proposed Aff-Wild2 containing the above three representations in the wild. There are various challenges in this dataset, such as head poses, ages, sex, etc.…”

Section: Introductionmentioning

confidence: 99%

Spatial-temporal Transformer for Affective Behavior Analysis

Zou¹,

Wang²,

Wen³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The utilization of multimodal features, including visual, audio, and text features, has been extensively employed in previous ABAW competitions (Zafeiriou et al 2017;Kollias, Sharmanska, and Zafeiriou 2019;Kollias and Zafeiriou 2021a,b;Kollias, Sharmanska, and Zafeiriou 2021;Kollias 2022Kollias , 2023Kollias et al 2023). We can improve the performance in affective behavior analysis tasks by extracting and analyzing these multimodal features.…”

Section: Related Work Multimodal Featuresmentioning

confidence: 99%

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Zhang¹,

An²,

Zishun³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this paper, we present our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes four sub-challenges of Valence-Arousal (VA) Estimation, Expression (Expr) Classification, Action Unit (AU) Detection and Emotional Reaction Intensity (ERI) Estimation. The 5th ABAW competition focuses on facial affect recognition utilizing different modalities and datasets. In our work, we extract powerful audio and visual features using a large number of sota models. These features are fused by Transformer Encoder and TEMMA. Besides, to avoid the possible impact of large dimensional differences between various features, we design an Affine Module to align different features to the same dimension. Extensive experiments demonstrate that the superiority of the proposed method. For the VA Estimation sub-challenge, our method obtains the mean Concordance Correlation Coefficient (CCC) of 0.6066. For the Expression Classification subchallenge, the average F1 Score is 0.4055. For the AU Detection sub-challenge, the average F1 Score is 0.5296. For the Emotional Reaction Intensity Estimation sub-challenge, the average pearson's correlations coefficient on the validation set is 0.3968. All of the results of four sub-challenges outperform the baseline with a large margin.

show abstract

“…From a machine learning perspective, AU detection in the wild presents many technical challenges. Most notably, in-the-wild datasets such as Aff-Wild2 [12][13][14][15][16][17][18][19][20][21]32] collect data with huge variations in the cameras (resulting in blurred video frames), environments (illumination conditions), and subjects (large variance in expressions, scale, and head poses). Ertugrul et al [4,5] demonstrate that the deep-learning-based AU detectors have limited generalization abilities due to the aforementioned variations.…”

Section: Introductionmentioning

confidence: 99%

Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild

Yin¹,

Tran²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Facial action unit detection has emerged as an important task within facial expression analysis, aimed at detecting specific pre-defined, objective facial expressions, such as lip tightening and cheek raising. This paper presents our submission to the Affective Behavior Analysis in-the-wild (ABAW) 2023 Competition for AU detection. We propose a multi-modal method for facial action unit detection with visual, acoustic, and lexical features extracted from the large pre-trained models. To provide high-quality details for visual feature extraction, we apply super-resolution and face alignment to the training data. Our approach achieves the F1 score of 52.3% on the official validation set.

show abstract

ABAW: Learning from Synthetic Data & Multi-task Learning Challenges

Cited by 35 publications

References 44 publications

Spatial-temporal Transformer for Affective Behavior Analysis

Spatial-temporal Transformer for Affective Behavior Analysis

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild

Contact Info

Product

Resources

About