Fine-grained Engagement Recognition in Online Learning Environment

Huang, Tao; Mei, Yunshan; Zhang, Hao; Liu, Sanya; Yang, Huali

doi:10.1109/iceiec.2019.8784559

Cited by 47 publications

(42 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparing the ResNet+TCN network with other works, we took reported results from the following methods: video-level and frame-level InceptionNet [7], C3D [7], I3D [16], DERN [13], and DFSTN [24]. In addition, we implemented the combination of the ResNet with LSTM, and C3D (up to the layer pool-5) [22] with LSTM (one-layer unidirectional with 128 hidden neurons) to investigate their performance compared to the ResNet+TCN method.…”

Section: Resultsmentioning

confidence: 99%

“…None of the previous works on the DAiSEE dataset, working with the original four-class annotations, reported their confusion matrices for test set [7], [16], [14], [13], [15], [24], and only reported the accuracy results. Therefore, it is hard to determine the individual performance of their methods on each of the engagement levels.…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, we presented a new end-to-end spatiotemporal hybrid architecture, ResNet+TCN, for determining the level of engagement among students in an online feature extraction [7], (d) C3D averaging + LSTM [30], (e) I3D [16], (f) ResNet + TCN with sampling and weighted loss (proposed), (g) C3D + LSTM [30], (h) LRCN [23], (i) C3D fine tuning [22], (j) DFSTN [24], (k) C3D + TCN (proposed), (l) DERN [13], (m) ResNet + LSTM (proposed), (n) ResNet + TCN (proposed). classroom setting.…”

Section: Discussionmentioning

confidence: 99%

“…Huang et al [13] proposed Deep Engagement Recognition Network (DERN) which combines bidirectional LSTM and attention mechanism to classify extracted features from faces and detect the level of engagement. They achieved 60% engagement level detection accuracy on the DAiSEE dataset.…”

Section: Literature Reviewmentioning

confidence: 99%

“…In traditional settings, in the video-based approaches, handcrafted features, such as eye gaze and head pose, can be extracted and classification algorithms can be trained to detect the level of engagement [7], [13], [14], [15], [16]. More recently, end-to-end video-based approaches have been proposed to detect student engagement, in which consecutive raw frames of video are fed to variants of Convolutional Neural Networks (CNNs) to detect the level of engagement [7], [13], [14], [15], [16].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Abedi¹,

Khan²

2021

Preprint

View full text Add to dashboard Cite

Automatic detection of students' engagement in online learning settings is a key element to improve the quality of learning and to deliver personalized learning materials to them. Varying levels of engagement exhibited by students in an online classroom is an affective behavior that takes place over space and time. Therefore, we formulate detecting levels of students' engagement from videos as a spatio-temporal classification problem. In this paper, we present a novel end-toend Residual Network (ResNet) and Temporal Convolutional Network (TCN) hybrid neural network architecture for students' engagement level detection in videos. The 2D ResNet extracts spatial features from consecutive video frames, and the TCN analyzes the temporal changes in video frames to detect the level of engagement. The spatial and temporal arms of the hybrid network are jointly trained on raw video frames of a large publicly available students' engagement detection dataset, DAiSEE. We compared our method with several competing students' engagement detection methods on this dataset. The ResNet+TCN architecture outperforms all other studied methods, improves the state-of-the-art engagement level detection accuracy, and sets a new baseline for future research.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Abedi¹,

Khan²

2021

Preprint

View full text Add to dashboard Cite

show abstract

Deep Learning Based Engagement Recognition in Highly Imbalanced Data

Dresvyanskiy

Minker

Karpov

2021

Speech and Computer

View full text Add to dashboard Cite

Engagement Detection with Multi-Task Training in E-Learning Environments

Copur,

Nakıp,

Scardapane

et al. 2022

Image Analysis and Processing – ICIAP 2022

View full text Add to dashboard Cite

Recognition of user interaction, in particular engagement detection, became highly crucial for online working and learning environments, especially during the COVID-19 outbreak. Such recognition and detection systems significantly improve the user experience and efficiency by providing valuable feedback. In this paper, we propose a novel Engagement Detection with Multi-Task Training (ED-MTT) system which minimizes mean squared error and triplet loss together to determine the engagement level of students in an e-learning environment. The performance of this system is evaluated and compared against the state-ofthe-art on a publicly available dataset as well as videos collected from real-life scenarios. The results show that ED-MTT achieves 6% lower MSE than the best state-of-the-art performance with highly acceptable training time and lightweight feature extraction.

show abstract

Fine-grained Engagement Recognition in Online Learning Environment

Cited by 47 publications

References 6 publications

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network

Deep Learning Based Engagement Recognition in Highly Imbalanced Data

Engagement Detection with Multi-Task Training in E-Learning Environments

Contact Info

Product

Resources

About