Sequencing educational content in classrooms using Bayesian knowledge tracing

Abstract

Despite the prevalence of e-learning systems in schools, most of today's systems do not personalize educational data to the individual needs of each student. This paper proposes a new algorithm for sequencing questions to students that is empirically shown to lead to better performance and engagement in real schools when compared to a baseline approach. It is based on using knowledge tracing to model students' skill acquisition over time, and to select questions that advance the student's learning within the range of the student's capabilities, as determined by the model. The algorithm is based on a Bayesian Knowledge Tracing (BKT) model that incorporates partial credit scores, reasoning about multiple attempts to solve problems, and integrating item difficulty. This model is shown to outperform other BKT models that do not reason about (or reason about some but not all) of these features. The model was incorporated into a sequencing algorithm and deployed in two classes in different schools where it was compared to a baseline sequencing algorithm that was designed by pedagogical experts. In both classes, students using the BKT sequencing approach solved more difficult questions and attributed higher performance than did students who used the expert-based approach. Students were also more engaged using the BKT approach, as determined by their interaction time and number of log-ins to the system, as well as their reported opinion. We expect our approach to inform the design of better methods for sequencing and personalizing educational content to students that will meet their individual learning needs.

Embracing imperfection in learning analytics

“…The results from Section 2 give us reason to pause. More convincing advances are provided by David et al [14], and the already discussed Liu and Koedinger [30]. These two papers back up their computational validation with user trials that demonstrate improved learning outcomes for students using the new tool over a comparable baseline.…”

Where is the Learning in Learning Analytics? A Systematic Literature Review on the Operationalization of Learning-Related Constructs in the Evaluation of Learning Analytics Interventions

“…A learner can demonstrate the acquisition of knowledge and skills as a product of their learning process. Most key studies in this category operationalize affected learning through grades or test scores or scores [29], [33]- [35], [37], [49], [50], [53], [54], [57], [61], [63], [67]- [70], [74]- [76], which is a direct assessment of learning as performance on a task (e.g., an exam or final test) [94]. Although grades may seem to be a direct operational definition, this is debatable.…”

Combining Difficulty Ranking with Multi-Armed Bandits to Sequence Educational Content

“…The students were randomly divided into 3 cohorts: (1) MAPLE Sequencing: Students in this group received questioned sequenced by the MAPLE algorithm when practicing with the system. (2) YBKT Sequencing 1 : students in this group received questioned sequenced by the Bayesian Knowledge Trace based algorithm proposed by Ben David et al [6]. This sequencing approach always chooses the next question from the available skill set in a deterministic manner.…”

“…We then implemented MAPLE in the wild in an existing e-learning system in a school with 7th grade students. MAPLE's performance was compared to two other sequencing algorithms already implemented in the e-learning system: an approach that sequenced questions according to educational expert guidelines and a state of the art Bayesian Knowledge Tracing based algorithm [6]. We found that our proposed approach showed promising results compared to the existing educational expert approach and the BKT based approach.…”