Facial Affect Recognition Using Semi-supervised Learning with Adaptive Threshold

Gera, Darshan; Kumar, Bobbili Veerendra Raj; Badveeti, Naveen Siva Kumar; Subramanian, Balasubramanian

doi:10.1007/978-3-031-25075-0_3

Cited by 1 publication

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As one can see, smoothing does not work for AU detection but can significantly improve the results for other tasks: up to 0.06 difference in F1-score for EXPR classification and up to 0.06 difference in mean CCC for VA prediction. Moreover, the smoothing works nicely even for blending the best ---0.981 SMMEmotionNet [23] 0.3648 0.2617 0.4737 1.1002 Two-Aspect Information Interaction [31] 0.515 0.207 0.385 1.107 SS-MFAR [4] 0.397 0.235 0.493 1.125 EfficientNet-B2 [27] 0.384 0.302 0.461 1.147 MAE+ViT [20] 0.4588 0.3028 0.5054 1.2671 Cross-attentive module [23] 0.499 0.333 0.456 1.288 MT-EmotiEffNet + OpenFace [29] 0.447 0.357 0.496 1.300 MAE+Transformer [37] 0 frame-level 0.4847 0.3578 0.5194 1.3619 + MT-EmotiDDAMFN smoothing, tAU = 0.5 0.5578 0.4168 0.5194 1.4939 models (Fig. 5).…”

Section: Multi-task Learning Challengementioning

confidence: 99%

See 1 more Smart Citation

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Savchenko¹

2021

2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY)

113

View full text Add to dashboard Cite

In this paper, we describe the results of the HSEmotion team in two tasks of the seventh Affective Behavior Analysis in-thewild (ABAW) competition, namely, multi-task learning for simultaneous prediction of facial expression, valence, arousal, and detection of action units, and compound expression recognition. We propose an efficient pipeline based on frame-level facial feature extractors pre-trained in multi-task settings to estimate valence-arousal and basic facial expressions given a facial photo. We ensure the privacy-awareness of our techniques by using the lightweight architectures of neural networks, such as MT-EmotiDDAMFN, MT-EmotiEffNet, and MT-EmotiMobileFaceNet, that can run even on a mobile device without the need to send facial video to a remote server. It was demonstrated that a significant step in improving the overall accuracy is the smoothing of neural network output scores using Gaussian or box filters. It was experimentally demonstrated that such a simple post-processing of predictions from simple blending of two top visual models improves the F1-score of facial expression recognition up to 7%. At the same time, the mean Concordance Correlation Coefficient (CCC) of valence and arousal is increased by up to 1.25 times compared to each model's frame-level predictions. As a result, our final performance score on the validation set from the multi-task learning challenge is 4.5 times higher than the baseline (1.494 vs 0.32).

show abstract

Section: Multi-task Learning Challengementioning

confidence: 99%

“…The two-aspect information interaction model [31] represents interactions between sign vehicles and messages. The SS-MFAR [4] extracts facial features using ResNet and leverages adaptive threshold for every class of facial expressions. The thresholds were estimated based on semi-supervised learning.…”

Section: Related Workmentioning

confidence: 99%

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Savchenko¹

2021

2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY)

113

View full text Add to dashboard Cite

show abstract

Facial Affect Recognition Using Semi-supervised Learning with Adaptive Threshold

Cited by 1 publication

References 23 publications

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Contact Info

Product

Resources

About