Multi-rate Attention Based GRU Model for Engagement Prediction

Zhu, Bin; Lan, Xinjie; Guo, Xin; Barner, Kenneth E.; Boncelet, Charles G.

doi:10.1145/3382507.3417965

Cited by 27 publications

(29 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, following [86], we utilized OpenFace [6] to extract the facial features including Facial Action Units related features (AUs) [29], Eye Gaze related features, and Head Pose related features (more details about features can be seen in [86]). Then, considering the robustness and efciency in computing, we trained a Random Forest Regressor model with 200 estimators/trees and achieved a 0.05 MSE score on the validation set (comparable with the SOTA models [92,106]).…”

Section: Student End: Learning Status Detectionmentioning

confidence: 99%

Glancee: An Adaptable System for Instructors to Grasp Student Learning Status in Synchronous Online Classes

Zhou

Nie

et al. 2022

CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

Section: Student End: Learning Status Detectionmentioning

confidence: 99%

Glancee: An Adaptable System for Instructors to Grasp Student Learning Status in Synchronous Online Classes

Zhou

Nie

et al. 2022

CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…In the feature-based engagement detection approaches, firstly, multi-modal handcrafted features are extracted from video/image and then fed to a classifier or regressor to detect the level of engagement in video/image [10], [6], [20], [21]. Wu et al in [20] proposed a feature-based approach for student's engagement level detection in EmotiW dataset [6].…”

Section: Literature Reviewmentioning

confidence: 99%

“…They extracted facial and upper-body features from videos and classified the features using a combination of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) to detect the level of engagement. Zhu et al in [21] proposed an attention-based GRU model to classify hand-crafted face and body features from videos and detect the level of engagement in the EmotiW dataset [6]. Whitehill et al in [5] proposed different feature-extraction (box filters and Gabor features) and classification (SVM and GentleBoost) combinations to detect the level of engagement of students from single images in their dataset.…”

Section: Literature Reviewmentioning

confidence: 99%

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Abedi¹,

Khan²

2021

Preprint

View full text Add to dashboard Cite

Automatic detection of students' engagement in online learning settings is a key element to improve the quality of learning and to deliver personalized learning materials to them. Varying levels of engagement exhibited by students in an online classroom is an affective behavior that takes place over space and time. Therefore, we formulate detecting levels of students' engagement from videos as a spatio-temporal classification problem. In this paper, we present a novel end-toend Residual Network (ResNet) and Temporal Convolutional Network (TCN) hybrid neural network architecture for students' engagement level detection in videos. The 2D ResNet extracts spatial features from consecutive video frames, and the TCN analyzes the temporal changes in video frames to detect the level of engagement. The spatial and temporal arms of the hybrid network are jointly trained on raw video frames of a large publicly available students' engagement detection dataset, DAiSEE. We compared our method with several competing students' engagement detection methods on this dataset. The ResNet+TCN architecture outperforms all other studied methods, improves the state-of-the-art engagement level detection accuracy, and sets a new baseline for future research.

show abstract

“…This makes the problem non-trivial and subjective because annotators can perceive different engagement levels from the same input video. The reliability of the dataset labels is a big concern in this setting but often is ignored by the current methods [29,30,32]. Because of this, deep learning models overfit to the uncertain samples and perform poorly on validation and test sets.…”

Section: Introductionmentioning

confidence: 99%

“…In our experimental work, we first analyze the importance of feature sets to select the best set of features for the resulting trained ED-MTT system. Then, we compare the performance of ED-MTT with 9 different works [1,5,15,20,24,25,27,31,32] from the state-of-the-art which will be reviewed in the next section. Our results show that ED-MTT outperforms these state-of-the-art methods with at least 6% improvement on MSE.…”

Section: Introductionmentioning

confidence: 99%

Engagement Detection with Multi-Task Training in E-Learning Environments

Çopur¹,

Nakıp²,

Scardapane³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recognition of user interaction, in particular engagement detection, became highly crucial for online working and learning environments, especially during the COVID-19 outbreak. Such recognition and detection systems significantly improve the user experience and efficiency by providing valuable feedback. In this paper, we propose a novel Engagement Detection with Multi-Task Training (ED-MTT) system which minimizes mean squared error and triplet loss together to determine the engagement level of students in an e-learning environment. The performance of this system is evaluated and compared against the state-ofthe-art on a publicly available dataset as well as videos collected from real-life scenarios. The results show that ED-MTT achieves 6% lower MSE than the best state-of-the-art performance with highly acceptable training time and lightweight feature extraction.

show abstract

Multi-rate Attention Based GRU Model for Engagement Prediction

Cited by 27 publications

References 5 publications

Glancee: An Adaptable System for Instructors to Grasp Student Learning Status in Synchronous Online Classes

Glancee: An Adaptable System for Instructors to Grasp Student Learning Status in Synchronous Online Classes

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Engagement Detection with Multi-Task Training in E-Learning Environments

Contact Info

Product

Resources

About