The FaceChannel: A Light-weight Deep Neural Network for Facial Expression Recognition

Barros, Pablo; Churamani, Nikhil; Sciutti, Alessandra

doi:10.1109/fg47880.2020.00070

Cited by 20 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our research, we also target the question of how machine learning is a reliable tool for the automatic extraction of affective states. In the last decade, many approaches primarily based on deep learning rely on pretraining facial images and emotion expressions on large datasets (Baltrusaitis et al, 2018 ; Barros et al, 2020 ). The available software seems promising for HRI tasks as it potentially enables researchers from other than ML areas to use those tools to explain higher-level human phenomena.…”

Section: Discussion and Limitations Of Affective State Recognitionmentioning

confidence: 99%

“…The neutral state is represented by (0, 0). For the extraction of arousal and valence, we employed the "FaceChannel" deep neural network architecture introduced by Barros et al (2020). In the FaceChannel network, each image or video frame is passed through a cascade of convolutional and pooling layers including an inhibition mechanism before passing to the classification stage.…”

Section: The Arousal-valence Dimensionmentioning

confidence: 99%

See 1 more Smart Citation

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

et al. 2022

View full text Add to dashboard Cite

A key goal in human-robot interaction (HRI) is to design scenarios between humanoid robots and humans such that the interaction is perceived as collaborative and natural, yet safe and comfortable for the human. Human skills like verbal and non-verbal communication are essential elements as humans tend to attribute social behaviors to robots. However, aspects like the uncanny valley and different technical affinity levels can impede the success of HRI scenarios, which has consequences on the establishment of long-term interaction qualities like trust and rapport. In the present study, we investigate the impact of a humanoid robot on human emotional responses during the performance of a cognitively demanding task. We set up three different conditions for the robot with increasing levels of social cue expressions in a between-group study design. For the analysis of emotions, we consider the eye gaze behavior, arousal-valence for affective states, and the detection of action units. Our analysis reveals that the participants display a high tendency toward positive emotions in presence of a robot with clear social skills compared to other conditions, where we show how emotions occur only at task onset. Our study also shows how different expression levels influence the analysis of the robots' role in HRI. Finally, we critically discuss the current trend of automatized emotion or affective state recognition in HRI and demonstrate issues that have direct consequences on the interpretation and, therefore, claims about human emotions in HRI studies.

show abstract

Section: Discussion and Limitations Of Affective State Recognitionmentioning

confidence: 99%

Section: The Arousal-valence Dimensionmentioning

confidence: 99%

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

et al. 2022

View full text Add to dashboard Cite

show abstract

“…While several state-of-the-art face detection approaches have taken into consideration general computer vision difficulties described above, there is another added challenge when attempting to detect the face of a neonate compared to the adult human face. Face-Channel [4] presents a lightweight deep learning approach of adult face detection. [3] is one of the first approaches specifically focusing of neonate face detection but looks to solve problem of pose variations.…”

Section: Motivation and Challengesmentioning

confidence: 99%

Robust Neonatal Face Detection in Real-world Clinical Settings

Hausmann¹,

Salekin²,

Goldgof³

et al. 2022

Preprint

View full text Add to dashboard Cite

Current face detection algorithms are extremely generalized and can obtain decent accuracy when detecting the adult faces. These approaches are insufficient when handling outlier cases, for example when trying to detect the face of a neonate infant whose face composition and expressions are relatively different than that of the adult. It is furthermore difficult when applied to detect faces in a complicated setting such as the Neonate Intensive Care Unit. By training a state-of-the-art face detection model, You-Only-Look-Once, on a proprietary dataset containing labelled neonate faces in a clinical setting, this work achieves near real time neonate face detection. Our preliminary findings show an accuracy of 68.7%, compared to the off the shelf solution which detected neonate faces with an accuracy of 7.37%. Although further experiments are needed to validate our model, our results are promising and prove the feasibility of detecting neonatal faces in challenging real-world settings. The robust and real-time detection of neonatal faces would benefit wide range of automated systems (e.g., pain recognition and surveillance) who currently suffer from the time and effort due to the necessity of manual annotations.To benefit the research community, we make our trained weights publicly available at github 1 .

show abstract

“…The robot uses its evaluation of participants' affective behaviour to model state-transitions, personalising interactions to individual preferences and employs Natural Language Generation (NLG) to generate naturalistic responses towards the participants. To evaluate the contribution of CL-based personalisation towards how the participants' impressions of the robot, we implement another version of the framework, for comparison, switching the CL-based model with the FaceChannel [3,4], an off-the-shelf state-of-the-art facial affect perception model that still encodes the participants' facial affect, albeit without any personalisation. Furthermore, a control condition is implemented by completely 'switching off' affective adaptation and always following a static and scripted interaction flow, ignoring the participants' affective responses.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Continual Learning for Affective Robotics: A Proof of Concept for Wellbeing

Churamani¹,

Axelsson²,

Caldir³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Sustaining real-world human-robot interactions requires robots to be sensitive to human behavioural idiosyncrasies and adapt their perception and behaviour models to cater to these individual preferences. For affective robots, this entails learning to adapt to individual affective behaviour to offer a personalised interaction experience to each individual. Continual Learning (CL) has been shown to enable real-time adaptation in agents, allowing them to learn with incrementally acquired data while preserving past knowledge. In this work, we present a novel framework for real-world application of CL for modelling personalised human-robot interactions using a CL-based affect perception mechanism. To evaluate the proposed framework, we undertake a proof-of-concept user study with 20 participants interacting with the Pepper robot using three variants of interaction behaviour: static and scripted, using affect-based adaptation without personalisation, and using affect-based adaptation with continual personalisation. Our results demonstrate a clear preference in the participants for CL-based continual personalisation with significant improvements observed in the robot's anthropomorphism, animacy and likeability ratings as well as the interactions being rated significantly higher for warmth and comfort as the robot is rated as significantly better at understanding how the participants feel.

show abstract

The FaceChannel: A Light-weight Deep Neural Network for Facial Expression Recognition

Cited by 20 publications

References 26 publications

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Robust Neonatal Face Detection in Real-world Clinical Settings

Continual Learning for Affective Robotics: A Proof of Concept for Wellbeing

Contact Info

Product

Resources

About