Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Abstract

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defned as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identifed which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose:(1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profle are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fxed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the * Both authors contributed equally to this paper.

Automatic Context-Aware Inference of Engagement in HMI: A Survey

“…Personalization with respect to culture, gender and each individual is performed using different fine-tuning strategies. In [190], personalized models of productive engagement for different learners' types were trained based on Efficient Neural Architecture Search [191].…”

“…As discussed in Section 3.3.2.2, a recent trend in engagement prediction systems is the development of personalised models [190], [196], [197]. In such frameworks, user profiling w.r.t.…”