Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System

Abstract

Assessment of students' self-regulated learning (SRL) requires a method for evaluating whether observed actions are appropriate acts of self-regulation in the specific learning context in which they occur. We review research that has resulted in an automated method for context-sensitive assessment of a specific SRL strategy, help seeking while working with an intelligent tutoring system. The method relies on a computer-executable model of the targeted SRL strategy. The method was validated by showing that it converges with other measures of help seeking. Automated feedback on help seeking driven by this method led to a lasting improvement in students' help-seeking behavior, although not in domain-specific learning. The method is unobtrusive, is temporally fine-grained, and can be applied on a large scale and over extended periods. The approach could be applied to other SRL strategies besides help seeking.

Addressing complexities in self-regulated learning: a focus on contextual factors, contingencies, and dynamic relations

“…Trace methodologies from think aloud protocols (Greene, Robertson, and Costa 2011) and sequenced, computer-generated logs of learner behaviors are increasingly common (Aleven et al 2010;Azevedo et al 2010;Graesser and McNamara. 2010;Greene and Azevedo 2010;Kinnebrew et al 2013a;Molenaar and Järvelä 2014;Sha, Looi, Chen, Seow, and Wong 2012;Winne and Hadwin 2013).…”

Examining self-efficacy during learning: variability and relations to behavior, performance, and learning

“…At the other end of this continuum are studies that select a small set of SRL processes and examine them closely in the context of a single learning task. These types of studies usually capture SRL events (Winne and Perry 2000) using process-oriented methodological approaches like microanalytic methods (Cleary 2011), think aloud protocols (Greene et al 2011), and trace methodologies (Aleven et al 2010;Azevedo et al 2013) to observe individual SRL processes as they unfold during learning.…”

Peer tutoring: old method, new developments / Tutoría entre iguales: método antiguo, nuevos avances

“…Removal of immediate feedback was associated with decreasing metacognitive performance, and this decline was not prevented when students were provided with other metacognitive scaffolds. Aleven, Roll, McLaren, and Koedinger (2010) sought to evaluate students' self-regulated learning (SRL) in the context of an intelligent tutoring system -a tricky proposition. An automated method for context-sensitive assessment of specific SRL strategies (e.g., help seeking) has been developed.…”