An evaluation of an adaptive learning system based on multimodal affect recognition for learners with intellectual disabilities

Abstract

Artificial intelligence tools for education (AIEd) have been used to automate the provision of learning support to mainstream learners. One of the most innovative approaches in this field is the use of data and machine learning for the detection of a student’s affective state, to move them out of negative states that inhibit learning, into positive states such as engagement. In spite of their obvious potential to provide the personalisation that would give extra support for learners with intellectual disabilities, little work on AIEd systems that utilise affect recognition currently addresses this group. Our system used multimodal sensor data and machine learning to first identify three affective states linked to learning (engagement, frustration, boredom) and second determine the presentation of learning content so that the learner is maintained in an optimal affective state and rate of learning is maximised. To evaluate this adaptive learning system, 67 participants aged between 6 and 18 years acting as their own control took part in a series of sessions using the system. Sessions alternated between using the system with both affect detection and learning achievement to drive the selection of learning content (intervention) and using learning achievement alone (control) to drive the selection of learning content. Lack of boredom was the state with the strongest link to achievement, with both frustration and engagement positively related to achievement. There was significantly more engagement and less boredom in intervention than control sessions, but no significant difference in achievement. These results suggest that engagement does increase when activities are tailored to the personal needs and emotional state of the learner and that the system was promoting affective states that in turn promote learning. However, longer exposure is necessary to determine the effect on learning.

Human Intelligence Analysis through Perception of AI in Teaching and Learning

“…In addition, the number of questions a student needs to answer to fully comprehend a given idea may be customized using these technological platforms. There is not a set of questions for each idea in advance [ 19 ]. To determine if a learner needs more practice with comparable problems or should go on to a more challenging level, the algorithm analyses their responses.…”

Affective domain studies of K-12 computing education: a systematic review from a perspective on affective objectives

“…The studies for designing teaching methods showed the roles and necessities of teaching procedures and interventions when applying ICT to education (Hwang & Yang, 2008;Martinez Borreguero et al, 2020;Repman, 1993;Woloshyn et al, 2017). For designing systems, their purposes were students' affect detection (Beege et al, 2018;Olugbade et al, 2020;Standen et al, 2020;Wu et al, 2014), reinforcement and feedback (Alepis et al, 2011;Rajendran et al, 2019;VanLehn et al, 2017), and both (Hwang & Yang, 2009). These studies found that environmental conditions for students' participation and flow were related to affective factors, so they attempted to collect and to utilize students' affective information.…”

“…First, in designing studies, outputs were systems and teaching methods. Systems were affective detectors (Richey et al, 2019;Standen et al, 2020;Wu et al, 2014), tutors for interaction between students and systems (VanLehn et al, 2017), and decision-making theory for affective judgment (Alepis et al, 2011). Teaching methods were instructional interventions that managed to change negative emotions in learning, such as frustration and boredom, into positive emotional responses and attitudes (Martinez Borreguero et al, 2020;Rajendran et al, 2019).…”

“…Third, according to international studies, differences in nationality do not imply differences in effectiveness. Participants in studies of Clark et al (2011), Nygren et al (2019 and Standen et al (2020) were from more than two countries. The studies found that contrary to expectations, effectiveness was unaffected by nationality.…”

Adaptive Instructional Systems. Design and Evaluation

“…The meta-analysis by Wu, Huang & Hwang [13] did report on type of participant yet did not identify any with special needs. Given the potential of affect sensitive adaptive learning systems to provide personalised support, school-aged students with learning disabilities and autism were identified as a stakeholder group of the MaTHiSiS project [14] which aimed to use affective state and performance to drive the presentation of learning material in an adaptive learning system.…”