Emotions and gesture recognition using affective computing assessment with deep learning

Abstract

Emotions have an important role in education. Affective development, attitudes, and emotions in learning are measured using affective assessment. This method is the right way to determine the student’s affective development. However, the process did not run optimally because the teacher found it difficult to collect student’s affective data. This paper describes the development of a system that can assist teachers in carrying out affective assessment. The system was developed using a v-model that aligns the verification phase with the validation. The use of the system is carried out during learning activities. The emotion detection system detects through body gestures using PoseNet to generate emotional data for each student. The detection results are then processed and displayed on an information system in the form of a website for affective assessment. The accuracy of emotion detection got validation values of 84.4% and 80.95% after being tested at school. In addition, the acceptance test with the usability aspect of the system by the teacher got a score of 77.56% and a score of 79.85% by the students. Based on several tests carried out, this developed system can assist the process of implementing affective assessment.

Convolutional neural network hyperparameters for face emotion recognition using genetic algorithm

“…The face plays a crucial role in expressing emotions such as happiness, sadness, anger, and others [1]. Compared to voice and other body movements, the face predominantly conveys a person's emotions [2].…”

Acoustic and visual geometry descriptor for multi-modal emotion recognition fromvideos

“…The emotion detection scheme discovered via body gestures uses PoseNet to generate emotional data for every student. The recognition results are treated and exhibited on an information system as a website [15]. Using images from the internet will evaluate the models to discover the best model for recognizing human emotions and detecting faces [16].…”