Tractable POMDP representations for intelligent tutoring systems

Abstract

With Partially Observable Markov Decision Processes (POMDPs), Intelligent Tutoring Systems (ITSs) can model individual learners from limited evidence and plan ahead despite uncertainty. However, POMDPs need appropriate representations to become tractable in ITSs that model many learner features, such as mastery of individual skills or the presence of specific misconceptions. This article describes two POMDP representations-state queues and observation chains-that take advantage of ITS task properties and let POMDPs scale to represent over 100 independent learner features. A real-world military training problem is given as one example. A human study (n = 14) provides initial validation for the model construction. Finally, evaluating the experimental representations with simulated students helps predict their impact on ITS performance. The compressed representations can model a wide range of simulated problems with instructional efficacy equal to lossless representations. With improved tractability, POMDP ITSs can accommodate more numerous or more detailed learner states and inputs.

Reinforcement Learning in a POMDP Based Intelligent Tutoring System for Optimizing Teaching Strategies

“…The more recent work included [10]- [14]. The work had the common feature of using POMDP to handle uncertainties in teaching processes.…”

“…A core component in the technique was a set of approximate POMDP policies, for dealing with uncertainties. The work described in [14] was aimed at making POMDP solvers feasible for real-world problems. In the work, a POMDP was used to cope with uncertainties in learners' mental processes and plans, which were difficult to discern.…”

Bayesian Affect Control Theory

“…Factored models allow for conditional independence to be explicitly stated in the model. POMDPs have been used as models for many human-interactive domains, including for intelligent tutoring systems [10], [11], and for human assistance systems [12].…”

“…This would require more complex models of the transitions in X (e.g. goals and problem dimensions [10], [11]), of the observations of X (e.g. from sensors and sentiment mappings), and of the dependence of the sentiments on the state.…”

“…POMDPs have been used as models for intelligent tutoring systems [10], including with emotional states [11]. Bayesian networks and probabilistic models have also seen recent developments [17] based on appraisal theory [19].…”