Engagement Evaluation in a Virtual Learning Environment via Facial Expression Recognition and Self-Reports: A Preliminary Approach

Olivetti, Elena Carlotta; Violante, Maria Grazia; Vezzetti, Enrico; Marcolin, Federica; Eynard, Benoît

doi:10.3390/app10010314

Cited by 18 publications

(5 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Observing one of the facial cues, i.e., the facial expression, is the direct way to recognise student focus during learning [72]. Analysis of Facial expression determines the interest and behaviour of the students during the learning process [10] same as what happens in the physical classroom environment. The teacher can quickly determine whether the student is engaged or disengaged in a physical classroom environment by observing their activities and reaction to a particular topic.…”

Section: Introductionmentioning

confidence: 99%

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Gupta

Kumar

Tekchandani

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Due to the COVID-19 crisis, the education sector has been shifted to a virtual environment. Monitoring the engagement level and providing regular feedback during e-classes is one of the major concerns, as this facility lacks in the e-learning environment due to no physical observation of the teacher. According to present study, an engagement detection system to ensure that the students get immediate feedback during e-Learning. Our proposed engagement system analyses the student's behaviour throughout the e-Learning session. The proposed novel approach evaluates three modalities based on the student's behaviour, such as facial expression, eye blink count, and head movement, from the live video streams to predict student engagement in e-learning. The proposed system is implemented based on deep-learning approaches such as VGG-19 and ResNet-50 for facial emotion recognition and the facial landmark approach for eye-blinking and head movement detection. The results from different modalities (for which the algorithms are proposed) are combined to determine the EI (engagement index). Based on EI value, an engaged or disengaged state is predicted. The present study suggests that the proposed facial cues-based multimodal system accurately determines student engagement in real time. The experimental research achieved an accuracy of 92.58% and showed that the proposed engagement detection approach significantly outperforms the existing approaches.

show abstract

Section: Introductionmentioning

confidence: 99%

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Gupta

Kumar

Tekchandani

2023

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…This could indicate that the actual level of involvement perceived by these subjects was lower than as stated. By comparing the results obtained with those of the previous work on the evaluation of involvement in a virtual learning environment [43], the use of neural networks has led to an improvement of the method. In fact, the correspondence between UES and facial expressions has increased from 50% to 58%.…”

Section: Discussionmentioning

confidence: 92%

Questionnaires or Inner Feelings: Who Measures the Engagement Better?

et al. 2020

Self Cite

View full text Add to dashboard Cite

This work proposes an innovative method for evaluating users’ engagement, combining the User Engagement Scale (UES) questionnaire and a facial expression recognition (FER) system, active research topics of increasing interest in the human–computer interaction domain (HCI). The subject of the study is a 3D simulator that reproduces a virtual FabLab in which users can approach and learn 3D modeling software and 3D printing. During the interaction with the virtual environment, a structured-light camera acquires the face of the participant in real-time, to catch its spontaneous reactions and compare them with the answers to the UES closed-ended questions. FER methods allow overcoming some intrinsic limits in the adoption of questioning methods, such as the non-sincerity of the interviewees and the lack of correspondence with facial expressions and body language. A convolutional neural network (CNN) has been trained on the Bosphorus database (DB) to perform expression recognition and the classification of the video frames in three classes of engagement (deactivation, average activation, and activation) according to the model of emotion developed by Russell. The results show that the two methodologies can be integrated to evaluate user engagement, to combine weighted answers and spontaneous reactions and to increase knowledge for the design of the new product or service.

show abstract

“…In the present research, we studied students' conceptual understanding of triangles' congruence using a conceptual framework, where it is recommended to use other conceptual frameworks and other methodologies that are suggested to assess different aspects of students' learning. Examples are the commognition framework [55,56], which is suggested to assess sharing knowledge; facial expression recognition, which is suggested to assess engagement [57]; or multi-block deep learning, which is suggested to assess emotions [58].…”

Section: Discussionmentioning

confidence: 99%

Learning Congruent Triangles through Ethnomathematics: The Case of Students with Difficulties in Mathematics

Shahbari

Daher

2020

Applied Sciences

View full text Add to dashboard Cite

Ethnomathematics makes school mathematics more relevant and meaningful for students. The current research aims to study the effect of using ethnomathematics in the context of Islamic ornamentation on learning the topic of congruent triangles. To achieve this aim, 30 10th-grade students engaged in ethnomathematics by learning about congruent triangles using Islamic ornamentation. Data was gathered via (a) videotaping and transcribing students’ learning and (b) students answering two parallel questionnaires that included proof questions on the three congruence theorems. The students were required to answer one questionnaire before the learning process and one after it. The main results indicated that the students succeeded in constructing the concepts of congruence and congruent triangles via the ethnomathematics learning process. In addition, the students succeeded in arriving at and formulating the three congruence theorems. Moreover, findings obtained from the questionnaires indicated that the students improved their proving processes as a result of ethnomathematics-based learning. Furthermore, paired sample t-tests indicated significant differences between the students’ mean scores before and after the learning process.

show abstract

Engagement Evaluation in a Virtual Learning Environment via Facial Expression Recognition and Self-Reports: A Preliminary Approach

Cited by 18 publications

References 34 publications

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Questionnaires or Inner Feelings: Who Measures the Engagement Better?

Learning Congruent Triangles through Ethnomathematics: The Case of Students with Difficulties in Mathematics

Contact Info

Product

Resources

About