Optimizing Human Learning

Abstract

Spaced repetition is a technique for efficient memorization which uses repeated, spaced review of content to improve long-term retention. Can we find the optimal reviewing schedule to maximize the benefits of spaced repetition? In this paper, we introduce a novel, flexible representation of spaced repetition using the framework of marked temporal point processes and then address the above question as an optimal control problem for stochastic differential equations with jumps. For two well-known human memory models, we show that the optimal reviewing schedule is given by the recall probability of the content to be learned. As a result, we can then develop a simple, scalable online algorithm, Memorize, to sample the optimal reviewing times. Experiments on both synthetic and real data gathered from Duolingo, a popular language-learning online platform, show that our algorithm may be able to help learners memorize more effectively than alternatives.

Deep Reinforcement Learning of Marked Temporal Point Processes

“…It is well known in the psychology literature that repeated and temporally distributed reviewing of information aids long term memorization [14,16,19,18]. Following recent work in the machine learning literature [18,22,27], we will consider the following setting: an online learning platform needs to teach one student some number of items with varying difficulty, say, words from the vocabulary of a foreign language. To this aim, the platform interacts with the student during a studying period by asking her to review each item multiple times, i.e., show a word to the student, ask for its translation, and then show the correct answer.…”

“…Interestingly, the above setting has been recently studied from the point of view of stochastic optimal control [27], where the authors have derived the optimal scheduling algorithm for a set of items. However, their solution assumes that the difficulty of the items and the student model are known [24] and that the objective function-the reward-has a particular functional form which depends on the average recall probability over time (and not the actual sampled recall at test time).…”

“…Experimental setup. Since we cannot make real interventions in an online learning platform, we use data from Duolingo to fit a probabilistic student model, as reported in previous work [24,27], which we then use to simulate a student's performance over time (refer to Appendix E for further details on the student model). Here, the optimal policy p * A;θ = (λ * θ (t), m * θ (t)) comprises of a reviewing intensity function and a multinomial mark distribution.…”

“…In this context, a recent line of work [13,27,29,30,33,34] has exploited an alternative view of MTPPs as stochastic differential equations (SDEs) with jumps [10] to design online, adaptive interventions using stochastic optimal control. While this line of work has shown promise at enhancing the functioning of social and information systems, their wide spread use and deployment is precluded mainly by two drawbacks.…”

“…In contrast, previous works considered the policy to be a probability distribution or, more rarely, a deterministic function [4,9,28]. Finally, we apply our methodology to two different applications in personalized teaching [14,22,27] and viral marketing [12,25,29,33,34], respectively. For simple dynamics and objective functions, which allow for stochastic optimal control approaches, our method achieves a comparable performance even though it does not have access to the true underlying dynamics.…”