Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation

Pourmirzaei, Mahdi; Montazer, Gholam Ali; Esmaili, Farzaneh

doi:10.48550/arxiv.2105.06421

Cited by 11 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods Accuracy ESR-9 [38] 59.30 RAN [41] 59.50 EfficientFace [54] 59.89 SCN [40] 60.23 SL (B0) [33] 60.34 SL (B2) [33] 60.35 Ours 60.72 The case of MAFT(Hard) is to prove that only MAFT during stage of MAFT will make model hard to train. Hence, we alternate normal classification tasks and mask attention fine-tuning tasks in our implementation.…”

Section: Table 3: Accuracy Comparison On Affectnet-8mentioning

confidence: 99%

Blindfold Attention: Novel Mask Strategy for Facial Expression Recognition

Mao

et al. 2022

Proceedings of the 2022 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is a basic and crucial computer vision task of classifying emotional expressions from human faces images into various emotion categories such as happy, sad, surprised, scared, angry, etc. Recently, facial expression recognition based on deep learning has made great progress. However, no matter the weight initialization technology or the attention mechanism, the face recognition method based on deep learning hard to capture those visually insignificant but semantically important features. To aid above question, in this paper we present a novel Facial Expression Recognition training strategy consisting of two components: Memo Affinity Loss (MAL) and Mask Attention Fine Tuning (MAFT). MAL is a variant of center loss, which uses memory bank strategy as well as discriminative center. MAL widens the distance between different clusters and narrows the distance within each cluster. Therefore, the features extracted by CNN were comprehensive and independent, which produced a more robust model. MAFT is a strategy that blindfolds attention parts temporarily and forces the model to learn from other important regions of the input image. It's not only an augmenting technique, but also a novel fine-tuning approach. As we know, we are the first to apply the mask strategy to the attention part and use this strategy to fine-tune the models. Finally, to implement our ideas, we constructed a new network named Architecture Attention ResNet based on ResNet-18. Our methods are conceptually and practically simple, but receives superior results on popular public facial expression recognition benchmarks with 88.75% on RAF-DB, 65.17% on AffectNet-7, 60.72% on AffectNet-8. The code will open source soon.

show abstract

Section: Table 3: Accuracy Comparison On Affectnet-8mentioning

confidence: 99%

Blindfold Attention: Novel Mask Strategy for Facial Expression Recognition

Mao

et al. 2022

Proceedings of the 2022 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…The best value of the weight hyperparameter w ∈ [0, 1], is estimated by maximizing the average F1 score on the validation set. [18] 61.32 ----SL + SSL inpanting-pl (B0) [16] 61.72 ----Distract Your Attention [22] 62.09 ----EfficientNet-B2 [19] 63.03 ----MT-EmotiEffNet 61.93 0.434 0.594 0.387 0.549…”

Section: Learning From Synthetic Data Challengementioning

confidence: 99%

HSE-NN Team at the 4th ABAW Competition: Multi-task Emotion Recognition and Learning from Synthetic Images

Savchenko¹

2022

Preprint

View full text Add to dashboard Cite

In this paper, we present the results of the HSE-NN team in the 4th competition on Affective Behavior Analysis in-the-wild (ABAW). The novel multi-task EfficientNet model is trained for simultaneous recognition of facial expressions and prediction of valence and arousal on static photos. The resulting MT-EmotiEffNet extracts visual features that are fed into simple feed-forward neural networks in the multi-task learning challenge. We obtain performance measure 1.3 on the validation set, which is significantly greater when compared to either performance of baseline (0.3) or existing models that are trained only on the s-Aff-Wild2 database. In the learning from synthetic data challenge, the quality of the original synthetic training set is increased by using the super-resolution techniques, such as Real-ESRGAN. Next, the MT-EmotiEffNet is finetuned on the new training set. The final prediction is a simple blending ensemble of pre-trained and fine-tuned MT-EmotiEffNets. Our average validation F1 score is 18% greater than the baseline convolutional neural network.

show abstract

“…In the context of FER, which is the focus of this paper, selfsupervised or contrastive learning have rarely been explored. In a recent work, FER was performed with a combination of contrastive learning and rotation prediction as pretext tasks [30].…”

Section: Self-supervised and Contrastive Learningmentioning

confidence: 99%

“…Despite the recent progress in FER systems with deep learning [21,22,30], most prior work in the area focus on only utilizing frontal facial views [16,30]. As a result, developed models may not be able to recognize expressions in faces captured from sharp side angles.…”

Section: Introductionmentioning

confidence: 99%

Self-supervised Contrastive Learning of Multi-view Facial Expressions

Etemad

2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Facial expression recognition (FER) has emerged as an important component of human-computer interaction systems. Despite recent advancements in FER, performance often drops significantly for non-frontal facial images. We propose Contrastive Learning of Multi-view facial Expressions (CL-MEx) to exploit facial images captured simultaneously from different angles towards FER. CL-MEx is a two-step training framework. In the first step, an encoder network is pre-trained with the proposed self-supervised contrastive loss, where it learns to generate view-invariant embeddings for different views of a subject. The model is then fine-tuned with labeled data in a supervised setting. We demonstrate the performance of the proposed method on two multi-view FER datasets, KDEF and DDCF, where state-of-the-art performances are achieved. Further experiments show the robustness of our method in dealing with challenging angles and reduced amounts of labeled data.

show abstract

Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation

Cited by 11 publications

References 0 publications

Blindfold Attention: Novel Mask Strategy for Facial Expression Recognition

Blindfold Attention: Novel Mask Strategy for Facial Expression Recognition

HSE-NN Team at the 4th ABAW Competition: Multi-task Emotion Recognition and Learning from Synthetic Images

Self-supervised Contrastive Learning of Multi-view Facial Expressions

Contact Info

Product

Resources

About