A Survey of AI-Based Facial Emotion Recognition: Features, ML &amp; DL Techniques, Age-Wise Datasets and Future Directions

Dalvi, Chirag; Rathod, Manish; Patil, Shruti; Gite, Shilpa; Kotecha, Ketan

doi:10.1109/access.2021.3131733

Cited by 45 publications

(22 citation statements)

References 176 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Now, although there have been different implementations carried out on kids’ emotion dataset, it was also observed that there is a scarcity of such dataset as there are not as many datasets of kids as compared with adults, which is indicated in a latest study [ 1 ]. It is also observed that, although the datasets of kids’ emotion are few, the individual quantity is less but the emotions recorded are also different as shown in Table 2 .…”

Section: Literature Reviewmentioning

confidence: 99%

“…After the literature review, the authors observed a scarcity in kids’ emotion datasets and saw that most of these datasets have not even achieved more than 75% accuracy after applying various approaches based on a latest study [ 1 ]. To challenge this, the authors created their own dataset and used the LIRIS dataset as well, and they also created separate versions of the datasets called LIRIS-Mesh and Authors-Mesh by using 3D landmark points, which were later experimented with various deep CNN models; the authors performed a comparative analysis of all the dataset types using the wide array of CNNs used.…”

Section: Proposed Modelmentioning

confidence: 99%

“…These emotions are depicted, most of the time, visually followed by the use of audio and language. In a recent study [ 1 ], according to a psychologist, visual factors contribute to 55% of the emotional understanding; audio contributes 38%, which usually can be categorized according to rhythm, pitch, or tone; and language contributes 7%, which is subject to the language complexity of all the languages around the world.…”

Section: Introductionmentioning

confidence: 99%

“…With FER, one could take advantage of the most natural feedback to digital platforms and hardware technology in the form of emotions, which will help to understand and perceive any product, idea, or platform according to the audience view. These are complex emotions according to a study [ 1 ] that discussed about wheel emotion and observed that there are some emotions globally understood without any hindrance. These emotions include happy, sad, surprise, fear, anger, disgust, and neutral.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Kids’ Emotion Recognition Using Various Deep-Learning Models with Explainable AI

Rathod

Dalvi

Kaur

et al. 2022

Sensors

View full text Add to dashboard Cite

Human ideas and sentiments are mirrored in facial expressions. They give the spectator a plethora of social cues, such as the viewer’s focus of attention, intention, motivation, and mood, which can help develop better interactive solutions in online platforms. This could be helpful for children while teaching them, which could help in cultivating a better interactive connect between teachers and students, since there is an increasing trend toward the online education platform due to the COVID-19 pandemic. To solve this, the authors proposed kids’ emotion recognition based on visual cues in this research with a justified reasoning model of explainable AI. The authors used two datasets to work on this problem; the first is the LIRIS Children Spontaneous Facial Expression Video Database, and the second is an author-created novel dataset of emotions displayed by children aged 7 to 10. The authors identified that the LIRIS dataset has achieved only 75% accuracy, and no study has worked further on this dataset in which the authors have achieved the highest accuracy of 89.31% and, in the authors’ dataset, an accuracy of 90.98%. The authors also realized that the face construction of children and adults is different, and the way children show emotions is very different and does not always follow the same way of facial expression for a specific emotion as compared with adults. Hence, the authors used 3D 468 landmark points and created two separate versions of the dataset from the original selected datasets, which are LIRIS-Mesh and Authors-Mesh. In total, all four types of datasets were used, namely LIRIS, the authors’ dataset, LIRIS-Mesh, and Authors-Mesh, and a comparative analysis was performed by using seven different CNN models. The authors not only compared all dataset types used on different CNN models but also explained for every type of CNN used on every specific dataset type how test images are perceived by the deep-learning models by using explainable artificial intelligence (XAI), which helps in localizing features contributing to particular emotions. The authors used three methods of XAI, namely Grad-CAM, Grad-CAM++, and SoftGrad, which help users further establish the appropriate reason for emotion detection by knowing the contribution of its features in it.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Proposed Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Kids’ Emotion Recognition Using Various Deep-Learning Models with Explainable AI

Rathod

Dalvi

Kaur

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In [ 31 ], the different stages of Facial Emotion Recognition (FER) such as pre-processing, feature extraction and classification using various methods and state-of-the-art CNN models are discussed. Comparison between different deep learning models, their benchmark accuracy and their architectural details are also discussed for model selection based on application and dataset.…”

Section: Introductionmentioning

confidence: 99%

A Survey on Physiological Signal-Based Emotion Recognition

Ahmad

Khan

2022

Bioengineering

View full text Add to dashboard Cite

Physiological signals are the most reliable form of signals for emotion recognition, as they cannot be controlled deliberately by the subject. Existing review papers on emotion recognition based on physiological signals surveyed only the regular steps involved in the workflow of emotion recognition such as pre-processing, feature extraction, and classification. While these are important steps, such steps are required for any signal processing application. Emotion recognition poses its own set of challenges that are very important to address for a robust system. Thus, to bridge the gap in the existing literature, in this paper, we review the effect of inter-subject data variance on emotion recognition, important data annotation techniques for emotion recognition and their comparison, data pre-processing techniques for each physiological signal, data splitting techniques for improving the generalization of emotion recognition models and different multimodal fusion techniques and their comparison. Finally, we discuss key challenges and future directions in this field.

show abstract

Berechenbare Gefühle? Grundlegendes zu einer Ethik der digitalen Emotionserfassung

Weber-Guskar

2023

Techno:Phil – Aktuelle Herausforderungen Der Technikphilosophie

View full text Add to dashboard Cite

A Survey of AI-Based Facial Emotion Recognition: Features, ML & DL Techniques, Age-Wise Datasets and Future Directions

Cited by 45 publications

References 176 publications

Kids’ Emotion Recognition Using Various Deep-Learning Models with Explainable AI

Kids’ Emotion Recognition Using Various Deep-Learning Models with Explainable AI

A Survey on Physiological Signal-Based Emotion Recognition

Berechenbare Gefühle? Grundlegendes zu einer Ethik der digitalen Emotionserfassung

Contact Info

Product

Resources

About