Capturing Session-to-Session Dynamics of Learning and Forgetting: Testing the Limits of Knowledge Tracing Models

Abstract

The development of intelligent tutoring systems and other educational technology necessitated the implementation and development of computational models of student learning. At the foundation of these models is the assumption that they accurately implement and track human cognitive processes. However, the extent to which this assumption is correct has not been fully tested. In the current paper, we use data from a large-scale longitudinal lab study to investigate the match between the processes instantiated in the models and human memory and learning processes. When fitted to the entire dataset retrospectively, the selected models of student learning (Bayesian Knowledge Tracing and Additive Factors Model) capture the qualitative trends of learning across sessions and relatively acceptable fit metrics. However, when the models are used to predict future behavior, as is often the goal in applied contexts, the picture changes considerably. We show that two of the most popular types of student learning models fail to account for basic cognitive principles—the spacing effect, and patterns of forgetting and learning across sessions. In fact, in some instances, having a poor model of human learning and memory may be worse than having no model of human learning and memory at all. At the same time, cognitive approaches can and should greatly benefit from computational modeling approaches; computational modeling offers opportunities for furthering both theory and practical impact.