Four-layer ConvNet to facial emotion recognition with minimal epochs and the significance of data diversity

Abstract

Emotion recognition is defined as identifying human emotion and is directly related to different fields such as human–computer interfaces, human emotional processing, irrational analysis, medical diagnostics, data-driven animation, human–robot communication, and many more. This paper proposes a new facial emotional recognition model using a convolutional neural network. Our proposed model, “ConvNet”, detects seven specific emotions from image data including anger, disgust, fear, happiness, neutrality, sadness, and surprise. The features extracted by the Local Binary Pattern (LBP), region based Oriented FAST and rotated BRIEF (ORB) and Convolutional Neural network (CNN) from facial expressions images were fused to develop the classification model through training by our proposed CNN model (ConvNet). Our method can converge quickly and achieves good performance which the authors can develop a real-time schema that can easily fit the model and sense emotions. Furthermore, this study focuses on the mental or emotional stuff of a man or woman using the behavioral aspects. To complete the training of the CNN network model, we use the FER2013 databases at first, and then apply the generalization techniques to the JAFFE and CK+ datasets respectively in the testing stage to evaluate the performance of the model. In the generalization approach on the JAFFE dataset, we get a 92.05% accuracy, while on the CK+ dataset, we acquire a 98.13% accuracy which achieve the best performance among existing methods. We also test the system’s success by identifying facial expressions in real-time. ConvNet consists of four layers of convolution together with two fully connected layers. The experimental results show that the ConvNet is able to achieve 96% training accuracy which is much better than current existing models. However, when compared to other validation methods, the suggested technique was more accurate. ConvNet also achieved validation accuracy of 91.01% for the FER2013 dataset. We also made all the materials publicly accessible for the research community at: https://github.com/Tanoy004/Emotion-recognition-through-CNN.

Federated learning-based AI approaches in smart healthcare: concepts, taxonomies, challenges and open issues

“…Two-stage federated Natural Language Processing (NLP) to detect patients and phenotyping from EHR data for obesity and comorbidities from several medical facilities is studied in [113]. Novel Federated Learning framework for smart wearables for activity recognition and data aggregation is studied in [114,115]. Model-centric, cross-device, horizontal, cross-silo, vertical, data-centric, reinforcement, and other types of Federated Learning exist.…”

An empirical study on the use of a facial emotion recognition system in guidance counseling utilizing the technology acceptance model and the general comfort questionnaire

“…This is an improvement to previous studies as it can reduce overfitting by employing a regularization method in fully connected layers. A more recent developed model ConvNet [19] which uses additional three layers, namely, Local Binary Pattern (LBP), Region Based Oriented FAST and rotated BRIEF (ORB), on top of CNN gained a higher accuracy of 98.13% using the same data set. This is the highest that was achieved using the said dataset even higher to a similar system which uses Extreme Learning Machine (ELM) universal approximation characteristic along with the Improved Black Hole algorithm which only achieved 90% [20].…”

“…While the two previous models [18,20] achieved good performance, it did not have a publicly available source code that can be used for the purpose of this study. Although [19] has a publicly available code in Python, it was not ready for integration. The model [21] was found to be publicly available in which a ready-for-integration JavaScript API [22] was also available.…”

“…Major breakthroughs have been made to significantly improve the performance of FER models. Although some of these models [19] were tested in real-time, there is very little study of their application in a real-world setting such as counselingan aspect in psychology which is heavily "invested" in emotions. Some research was done to investigate the use of an FER in entertainment [4], e-learning [5], politics [6] and even robotics [7], among others.…”

“…were considered. At least seven studies [10,11,[18][19][20][21]27] were initially considered as candidate models. However, in future studies, the researchers recommend the use of Scopus, the most comprehensive abstract database, to expand the list of candidate models.…”