Deep Learning Methods for Facial Expression Recognition

Refat, Chowdhury Mohammad Masum; Azlan, Norsinnira Zainul

doi:10.1109/icom47790.2019.8952056

Cited by 12 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They tested the proposed residual convolutional neural network model on the CK+ dataset and got a recognition accuracy of 97%. Refat and Azlan [47] compared the effects of DBN and CNN in the application of facial expression recognition and obtained that the classification effect of CNN was better from the aspects of training time and classification accuracy. The accuracy of the proposed system was 97.1%, with seven categories of total facial expressions, such as smile, sadness, anger, surprise, fear, disgust, and neutral.…”

Section: Related Workmentioning

confidence: 99%

DLSANet: Facial expression recognition with double‐code LBP‐layer spatial‐attention network

Guo

Wang

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Facial expression recognition (FER) is widely used in many fields. To further improve the accuracy of FER, this paper proposes a method based on double‐code LBP‐layer spatial‐attention network (DLSANet). The backbone model for the DLSANet is an emotion network (ENet), which is modified with a double‐code LBP (DLBP) layer and a spatial attention module. The DLBP layer is at the front of the first convolutional layer. More valuable features can be extracted by inputting the image processed by DLBP into convolutional layers. The JAFFE and CK+ datasets are used, which contain seven expressions: happiness, anger, disgust, neutral, fear, sadness, and surprise. The average of fivefold cross‐validation shows that DLSANet achieves a recognition accuracy of 93.81% and 98.68% on the JAFFE and CK+ datasets. The experiment reveals that the DLSANet can produce better classification results than state‐of‐the‐art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

DLSANet: Facial expression recognition with double‐code LBP‐layer spatial‐attention network

Guo

Wang

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…In contrast to this, the neural approaches focus on the application of Convolutional Neural Networks (CNNs) and generic image processing backbones to build often much more robust emotion detection algorithms. For several years, CNNs have been shown to provide highly accurate results in image analysis in emotion recognition ( Refat and Azlan, 2019 ).…”

Section: Related Workmentioning

confidence: 99%

Invoking and identifying task-oriented interlocutor confusion in human-robot interaction

Li,

Ross

2023

Front. Robot. AI

View full text Add to dashboard Cite

Successful conversational interaction with a social robot requires not only an assessment of a user’s contribution to an interaction, but also awareness of their emotional and attitudinal states as the interaction unfolds. To this end, our research aims to systematically trigger, but then interpret human behaviors to track different states of potential user confusion in interaction so that systems can be primed to adjust their policies in light of users entering confusion states. In this paper, we present a detailed human-robot interaction study to prompt, investigate, and eventually detect confusion states in users. The study itself employs a Wizard-of-Oz (WoZ) style design with a Pepper robot to prompt confusion states for task-oriented dialogues in a well-defined manner. The data collected from 81 participants includes audio and visual data, from both the robot’s perspective and the environment, as well as participant survey data. From these data, we evaluated the correlations of induced confusion conditions with multimodal data, including eye gaze estimation, head pose estimation, facial emotion detection, silence duration time, and user speech analysis—including emotion and pitch analysis. Analysis shows significant differences of participants’ behaviors in states of confusion based on these signals, as well as a strong correlation between confusion conditions and participants own self-reported confusion scores. The paper establishes strong correlations between confusion levels and these observable features, and lays the ground or a more complete social and affect oriented strategy for task-oriented human-robot interaction. The contributions of this paper include the methodology applied, dataset, and our systematic analysis.

show abstract

“…DL aims to develop end-to-end systems to reduce the dependency from hand-crafted features, pre-processing, and feature extraction techniques (Ghayoumi, 2017 ). Notably, convolutional neural networks (CNNs) have been proven to be particularly efficient in this task (Mollahosseini et al, 2017 ; Zhang, 2017 ; Refat and Azlan, 2019 ).…”

Section: State Of the Artmentioning

confidence: 99%

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

2020

View full text Add to dashboard Cite

A fascinating challenge in the field of human–robot interaction is the possibility to endow robots with emotional intelligence in order to make the interaction more intuitive, genuine, and natural. To achieve this, a critical point is the capability of the robot to infer and interpret human emotions. Emotion recognition has been widely explored in the broader fields of human–machine interaction and affective computing. Here, we report recent advances in emotion recognition, with particular regard to the human–robot interaction context. Our aim is to review the state of the art of currently adopted emotional models, interaction modalities, and classification strategies and offer our point of view on future developments and critical issues. We focus on facial expressions, body poses and kinematics, voice, brain activity, and peripheral physiological responses, also providing a list of available datasets containing data from these modalities.

show abstract

Deep Learning Methods for Facial Expression Recognition

Cited by 12 publications

References 16 publications

DLSANet: Facial expression recognition with double‐code LBP‐layer spatial‐attention network

DLSANet: Facial expression recognition with double‐code LBP‐layer spatial‐attention network

Invoking and identifying task-oriented interlocutor confusion in human-robot interaction

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

Contact Info

Product

Resources

About