Multitask Emotion Recognition Model with Knowledge Distillation and Task Discriminator

Jeong, Euiseok; Oh, Geesung; Lim, Sejoon

doi:10.48550/arxiv.2203.13072

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike the experimental setup in ABAW and ABAW2, where three different tasks were completed independently, ABAW3 presents an integrated metric and evaluates the performance of all three tasks simultaneously. Deng [3] employs psychological prior knowledge for multi-task estimation, which uses local features for AU recognition and merges the messages of different regions for EXPR and VA. Jeong et al [8] apply the knowledge distillation technique for a better generalization performance and the domain adaptation techniques to improve accuracy in target domains. Savchenko et al [10] used a lightweight EfficientNet model to develop a real-time framework and improve performance by pre-training based on additional data.…”

Section: Related Workmentioning

confidence: 99%

Affective Behaviour Analysis Using Pretrained Model with Facial Priori

Li¹,

Sun²,

Zhaori³

et al. 2022

Preprint

View full text Add to dashboard Cite

Affective behaviour analysis has aroused researchers' attention due to its broad applications. However, it is labor exhaustive to obtain accurate annotations for massive face images. Thus, we propose to utilize the prior facial information via Masked Auto-Encoder (MAE) pretrained on unlabeled face images. Furthermore, we combine MAE pretrained Vision Transformer (ViT) and AffectNet pretrained CNN to perform multi-task emotion recognition. We notice that expression and action unit (AU) scores are pure and intact features for valence-arousal (VA) regression. As a result, we utilize AffectNet pretrained CNN to extract expression scores concatenating with expression and AU scores from ViT to obtain the final VA features. Moreover, we also propose a co-training framework with two parallel MAE pretrained ViT for expression recognition tasks. In order to make the two views independent, we random mask most patches during the training process. Then, JS divergence is performed to make the predictions of the two views as consistent as possible. The results on ABAW4 show that our methods are effective.

show abstract

Section: Related Workmentioning

confidence: 99%