Action unit classification for facial expression recognition using active learning and SVM

Yao, Li; Wan, Yan; Ni, Hongjie; Xu, Bugao

doi:10.1007/s11042-021-10836-w

Cited by 42 publications

(16 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Action Unit (AU)-based representation the relation between facial feature and AUs; relation between AUs; relation between AUs and expressions need to be determined; hence, it is more complex in nature and difficult to implement by considering spatial information only. AU-based expression recognition methods demand action units whose data related to training has been previously labelled by experts [15]. This is a process which is quite time-consuming and labour-intensive.…”

Section: Related Workmentioning

confidence: 99%

Automatic facial expression recognition combining texture and shape features from prominent facial regions

Kumar¹,

Kumar

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

Facial expression is one form of communication which being non‐verbal in nature precedes verbal communication in both origin and conception. Most of the existing methods for Automatic Facial Expression Recognition (AFER) are mainly focused on global feature extraction assuming that all facial regions contribute equal amount of discriminative information to predict the expression class. The detection and localization of facial regions that have significant contribution to expression recognition and extraction of highly discriminative feature distribution from those regions are not fully explored. The key contributions of the proposed work are developing novel feature distribution upon combining the discriminative power of shape and texture feature; determining the contribution of facial regions and identifying the prominent facial regions that hold abstract and highly discriminative information for expression recognition. The shape and texture features taken into consideration are Local Phase Quantization (LPQ), Local Binary Pattern (LBP), and Histogram of Oriented Gradients (HOG). Multiclass Support Vector Machine (MSVM) is used while one versus one classification. The proposed work is implemented on CK+, KDEF, and JAFFE benchmark facial expression datasets. The recognition rate of the proposed work is 94.2% on CK+ and 93.7% on KDEF, which is significantly more than the existing handcrafted feature‐based methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic facial expression recognition combining texture and shape features from prominent facial regions

Kumar¹,

Kumar

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…Facial features that have been reported to be informative in classification of psychiatric disorders, especially depression, are facial expressions and face movement measures ( Schultebraucks et al, 2020b ; Stolicyn et al, 2020 ). Facial expressions used for prediction are in the literature described and supported by individual components of muscle movement, called Action Units (AUs) ( Yao et al, 2021 ). There are 23 AUs all together, but some have stood out more than others in predicting psychopathological states ( Gavrilescu and Vizireanu, 2019 ; Kumar et al, 2020 ).…”

Section: Predictionmentioning

confidence: 99%

AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients

et al. 2021

View full text Add to dashboard Cite

The COVID-19 pandemic has adverse consequences on human psychology and behavior long after initial recovery from the virus. These COVID-19 health sequelae, if undetected and left untreated, may lead to more enduring mental health problems, and put vulnerable individuals at risk of developing more serious psychopathologies. Therefore, an early distinction of such vulnerable individuals from those who are more resilient is important to undertake timely preventive interventions. The main aim of this article is to present a comprehensive multimodal conceptual approach for addressing these potential psychological and behavioral mental health changes using state-of-the-art tools and means of artificial intelligence (AI). Mental health COVID-19 recovery programs at post-COVID clinics based on AI prediction and prevention strategies may significantly improve the global mental health of ex-COVID-19 patients. Most COVID-19 recovery programs currently involve specialists such as pulmonologists, cardiologists, and neurologists, but there is a lack of psychiatrist care. The focus of this article is on new tools which can enhance the current limited psychiatrist resources and capabilities in coping with the upcoming challenges related to widespread mental health disorders. Patients affected by COVID-19 are more vulnerable to psychological and behavioral changes than non-COVID populations and therefore they deserve careful clinical psychological screening in post-COVID clinics. However, despite significant advances in research, the pace of progress in prevention of psychiatric disorders in these patients is still insufficient. Current approaches for the diagnosis of psychiatric disorders largely rely on clinical rating scales, as well as self-rating questionnaires that are inadequate for comprehensive assessment of ex-COVID-19 patients’ susceptibility to mental health deterioration. These limitations can presumably be overcome by applying state-of-the-art AI-based tools in diagnosis, prevention, and treatment of psychiatric disorders in acute phase of disease to prevent more chronic psychiatric consequences.

show abstract

“…In the work of Li et al [50], they used a first classifier to predict the presence of 14 AUs. Then, they fed the output of this classifier into an SVM to recognize emotions, reaching an average recognition rate of 94.07% for females and 90.77% for males on the CK database.…”

Section: Facial Emotion Recognitionmentioning

confidence: 99%

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

et al. 2021

View full text Add to dashboard Cite

Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated a pre-trained xlsr-Wav2Vec2.0 transformer using two transfer-learning techniques: embedding extraction and fine-tuning. The best accuracy results were achieved when we fine-tuned the whole model by appending a multilayer perceptron on top of it, confirming that the training was more robust when it did not start from scratch and the previous knowledge of the network was similar to the task to adapt. Regarding the facial emotion recognizer, we extracted the Action Units of the videos and compared the performance between employing static models against sequential models. Results showed that sequential models beat static models by a narrow difference. Error analysis reported that the visual systems could improve with a detector of high-emotional load frames, which opened a new line of research to discover new ways to learn from videos. Finally, combining these two modalities with a late fusion strategy, we achieved 86.70% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. Results demonstrated that these modalities carried relevant information to detect users’ emotional state and their combination allowed to improve the final system performance.

show abstract

Action unit classification for facial expression recognition using active learning and SVM

Cited by 42 publications

References 20 publications

Automatic facial expression recognition combining texture and shape features from prominent facial regions

Automatic facial expression recognition combining texture and shape features from prominent facial regions

AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

Contact Info

Product

Resources

About