Developing a Deep Learning-Based Affect Recognition System for Young Children

Farzaneh, Amir Hossein; Kim, Yanghee; Zhou, Mengxi; Qi, Xiaojun

doi:10.1007/978-3-030-23207-8_14

Cited by 6 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They designed a CNN model and trained it for classifying six basic emotions, including neutral emotion, in children and adults. Amir et al [33] developed an affect recognition system for young children using a deep convolutional neural network. Their DCNN is an emotion recognition prototype that effectively extracts refined facial features.…”

Section: Child Fer Datasetsmentioning

confidence: 99%

Recognition of Children’s Facial Expressions Using Deep Learned Features

Laraib,

Shaukat,

Khan

et al. 2023

Electronics

View full text Add to dashboard Cite

The emotional well-being of a child is crucial for their successful integration into society as a productive individual. While technology has made significant strides in enabling machines to decipher human emotional signals, current research in emotion recognition primarily prioritizes adults, disregarding the fact that children develop emotional awareness at an early stage. This highlights the need to explore how machines can recognize facial expressions in children, although the absence of a standardized database poses a challenge. In this study, we propose a system that employs Convolutional-Neural-Network (CNN)-based models, such as VGG19, VGG16, and Resnet50, as feature extractors, and Support Vector Machine (SVM) and Decision Tree (DT) for classification, to automatically recognize children’s expressions using a video dataset, namely Children’s Spontaneous Facial Expressions (LIRIS-CSE). Our system is evaluated through various experimental setups, including 80–20% split, K-Fold Cross-Validation (K-Fold CV), and leave one out cross-validation (LOOCV), for both image-based and video-based classification. Remarkably, our research achieves a promising classification accuracy of 99% for image-based classification, utilizing features from all three networks with SVM using 80–20% split and K-Fold CV. For video-based classification, we achieve 94% accuracy using features from VGG19 with SVM using LOOCV. These results surpass the performance of the original work, which reported an average image-based classification accuracy of 75% on their LIRIS-CSE dataset. The favorable outcomes obtained from our research can pave the way for the practical application of our proposed emotion recognition methodology in real-world scenarios.

show abstract

Section: Child Fer Datasetsmentioning

confidence: 99%

Recognition of Children’s Facial Expressions Using Deep Learned Features

Laraib,

Shaukat,

Khan

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…The multiple reviews on the affect recognition field, e.g., [ 8 , 26 , 27 , 28 , 29 ], are a clear indicator of the importance of the topic and the level of research activity that has been reached. Affect recognition has been successfully applied to marketing [ 30 , 31 ], health [ 32 , 33 ], cognitive assistants [ 34 ] and more recently learning systems [ 35 , 36 , 37 ], including methods to automatically recognise student’s engagement [ 38 ].…”

Section: Related Workmentioning

confidence: 99%

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts from Nonintrusive Physiological Signals in Real-World Situations

Serrano-Mamolar

Arevalillo‐Herráez

Chicote-Huete

et al. 2021

Sensors

View full text Add to dashboard Cite

Previous research has proven the strong influence of emotions on student engagement and motivation. Therefore, emotion recognition is becoming very relevant in educational scenarios, but there is no standard method for predicting students’ affects. However, physiological signals have been widely used in educational contexts. Some physiological signals have shown a high accuracy in detecting emotions because they reflect spontaneous affect-related information, which is fresh and does not require additional control or interpretation. Most proposed works use measuring equipment for which applicability in real-world scenarios is limited because of its high cost and intrusiveness. To tackle this problem, in this work, we analyse the feasibility of developing low-cost and nonintrusive devices to obtain a high detection accuracy from easy-to-capture signals. By using both inter-subject and intra-subject models, we present an experimental study that aims to explore the potential application of Hidden Markov Models (HMM) to predict the concentration state from 4 commonly used physiological signals, namely heart rate, breath rate, skin conductance and skin temperature. We also study the effect of combining these four signals and analyse their potential use in an educational context in terms of intrusiveness, cost and accuracy. The results show that a high accuracy can be achieved with three of the signals when using HMM-based intra-subject models. However, inter-subject models, which are meant to obtain subject-independent approaches for affect detection, fail at the same task.

show abstract

“…[8,[26][27][28][29], are a clear indicator of the importance of the topic and the level of research activity that has been reached. Affect recognition has been successfully applied to marketing [30,31], health [32,33] and more recently to learning systems [34,35], including methods to automatically recognise student's engagement [36].…”

Section: Related Workmentioning

confidence: 99%

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts From Non-Intrusive Physiological Signals Inreal-World Situations

Serrano-Mamolar¹,

Arevalillo‐Herráez²,

Boticario³

2021

Preprint

View full text Add to dashboard Cite

Emotion recognition is becoming very relevant in educational scenarios, since previous research has proven the strong influence of emotions on the student's engagement and motivation. There is no standard method for stating student's affect, but physiological signals have been widely used in educational contexts. Physiological signals have been proved to offer high accuracy in detecting emotions because they reflect spontaneous affect-related information, and which is fresh and do not require an additional control or interpretation. However, most proposed works use measuring equipment that limit its applicability in real-world scenarios because of their high cost and their intrusiveness. Expensive material means an economic challenge for schools and reduce the scalability of the experiments. Intrusive equipment can be uncomfortable for the students which can lead to errors in the collected data. In this work, we analyse the feasibility of developing a low-cost non-intrusive device that integrates easy-to-capture signals that guarantee high detection accuracy. The advantage of the approach also lies in using user’s centred information sources (intra-subject) in real-world situations, which provide better detection accuracy, by offering models adapted to each subject. To this end, we present an experimental study that aims to explore the potential application of Hidden Markov Models (HMM) to predict the concentration state from 4 commonly used physiological signals, namely heart rate, breath rate, skin conductance and skin temperature. We study the multi-fusion of every possible combination of these four signals and analyse their potential use in an educational context in terms of intrusiveness, cost and accuracy. Results show that a high accuracy can be achieved with three of the signals when using HMM-based intra-subject models. However, inter-subject models, which are meant to obtain subject-independent approaches for affect detection, fail at the same task. This work concludes that the implementation of a low-cost wrist-worn device for recognising relevant emotions from each student is possible and open the way to a wide range of practical applications in the context of adaptive learning systems.

show abstract

Developing a Deep Learning-Based Affect Recognition System for Young Children

Cited by 6 publications

References 17 publications

Recognition of Children’s Facial Expressions Using Deep Learned Features

Recognition of Children’s Facial Expressions Using Deep Learned Features

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts from Nonintrusive Physiological Signals in Real-World Situations

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts From Non-Intrusive Physiological Signals Inreal-World Situations

Contact Info

Product

Resources

About