Causal affect prediction model using a past facial image sequence

Oh, Geesung; Jeong, Euiseok; Lim, Sejoon

doi:10.1109/iccvw54120.2021.00395

Cited by 14 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A natural extension of the above would be to attempt to leverage causal inference for another computer vision task: facial expression recognition. Indeed, existing attempts at doing so have been highly successful [2,29]. However, existing works have only leveraged on sequential data input [29] or investigated the use of interventions, back-door adjustments, and confounders [2,3].…”

Section: Causality For Facial Affect Recognitionmentioning

confidence: 99%

“…Indeed, existing attempts at doing so have been highly successful [2,29]. However, existing works have only leveraged on sequential data input [29] or investigated the use of interventions, back-door adjustments, and confounders [2,3]. None of the existing works have explored the usage of structural causal models to formalise bias.…”

Section: Causality For Facial Affect Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Causal Structure Learning of Bias for Fair Affect Recognition

Cheong

Kalkan

Güneş

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)

View full text Add to dashboard Cite

The problem of bias in facial affect recognition tools can lead to severe consequences and issues. It has been posited that causality is able to address the gaps induced by the associational nature of traditional machine learning, and one such gap is that of fairness. However, given the nascency of the field, there is still no clear mapping between tools in causality and applications in fair machine learning for the specific task of affect recognition. To address this gap, we provide the first causal structure formalisation of the different biases that can arise in affect recognition. We conducted a proof of concept on utilising causal structure learning for the post-hoc understanding and analysing bias.

show abstract

Section: Causality For Facial Affect Recognitionmentioning

confidence: 99%

Section: Causality For Facial Affect Recognitionmentioning

confidence: 99%

Causal Structure Learning of Bias for Fair Affect Recognition

Cheong

Kalkan

Güneş

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)

View full text Add to dashboard Cite

show abstract

“…The overall loss is the sum of losses of each task. MFAR network minimizes the overall loss function 3 . 2.2 SS-MFAR : Semi-Supervised Multi-Task Facial Affect Recognition SS-MFAR architecture is shown in the Figure 2.…”

Section: Mfar : Multi-task Facial Affect Recognitionmentioning

confidence: 99%

“…Automatic affect recognition is currently an active area of research and has applications in many areas such as education, gaming, software development, auto motives [1,2], medical care, etc. Many works have dealt with Valence-arousal estimation [3,4,5,6,7], action unit detection [8,9,10], expression classification [] tasks individually. [11] introduced a framework which uses only static images and Multi-Task-Learning (MTL) to learn categorical representations and use them to estimate dimensional representation, but is limited to AffectNet data set [12].…”

Section: Introductionmentioning

confidence: 99%

SS-MFAR : Semi-supervised Multi-task Facial Affect Recognition

Gera¹,

Kumar²,

Kumar³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automatic affect recognition has applications in many areas such as education, gaming, software development, automotives, medical care, etc. but it is non trivial task to achieve appreciable performance on in-the-wild data sets. In-the-wild data sets though represent real-world scenarios better than synthetic data sets, the former ones suffer from the problem of incomplete labels. Inspired by semi-supervised learning, in this paper, we introduce our submission to the Multi-Task-Learning Challenge at the 4th Affective Behavior Analysis in-the-wild (ABAW) 2022 Competition. The three tasks that are considered in this challenge are valence-arousal(VA) estimation, classification of expressions into 6 basic (anger, disgust, fear, happiness, sadness, surprise), neutral, and the 'other' category and 12 action units(AU) numbered AU-{1,

show abstract

“…The input images are evenly time-divided into six equal parts and input to a face detector; the MMOD-based face detector outputs one cropped face image with the highest confidence value for each input. The cropped images are resized to the input shape of the feature extractor and sequentially fed into a feature extractor and a classifier based on CAPNet [41]. Because the classification form is different from that of CAPNet, only the number of units in the top layer of the classifier is modified to the number of representative driver emotional states.…”

mentioning

confidence: 99%

Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving

Jeong

Kim

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

As vehicles provide various services to drivers, research on driver emotion recognition has been expanding. However, current driver emotion datasets are limited by inconsistencies in collected data and inferred emotional state annotations by others. To overcome this limitation, we propose a data collection system that collects multimodal datasets during real-world driving. The proposed system includes a self-reportable HMI application into which a driver directly inputs their current emotion state. Data collection was completed without any accidents for over 122 h of real-world driving using the system, which also considers the minimization of behavioral and cognitive disturbances. To demonstrate the validity of our collected dataset, we also provide case studies for statistical analysis, driver face detection, and personalized driver emotion recognition. The proposed data collection system enables the construction of reliable large-scale datasets on real-world driving and facilitates research on driver emotion recognition. The proposed system is avaliable on GitHub.

show abstract

Causal affect prediction model using a past facial image sequence

Cited by 14 publications

References 14 publications

Causal Structure Learning of Bias for Fair Affect Recognition

Causal Structure Learning of Bias for Fair Affect Recognition

SS-MFAR : Semi-supervised Multi-task Facial Affect Recognition

Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving

Contact Info

Product

Resources

About