Experimental Evaluation of Automatic Hint Generation for a Logic Tutor

Adaptive structure metrics for automated feedback provision in intelligent tutoring systems

“…The systems job is to select the next task depending on the current level of knowledge and individual parameters of the student (outer loop) and to support her solving the current task (inner loop) [5]. Such systems have been successfully applied in many contexts, especially in learning logic and math concepts, and have been proven to lead to positive learning outcomes for students [12,6]. However, they usually rely on extensive knowledge engineering to formalize domain concepts and explicitly track student knowledge, which is both costly and difficult, especially in domains where explicit and detailed knowledge about the domain can not be obtained (so-called ill-defined domains) [3,4,5].…”

“…To relieve ITS engineers from the burden of knowledge engineering, data-driven approaches have emerged. Such approaches try to replace pre-defined and explicit domain knowledge by inference based on example-data of students interacting with the system [5,6].…”

“…However, researchers have reported 100 -1,000 hours of authoring time for one hour of instructions in ITSs [3]; in addition, ITSs usually require an underlying domain theory such that their applicability is limited in areas where problems and their solution strategies are not easy to formalize [4,5]. In such domains, data-driven approaches are possible, providing feedback based on a set of existing examples for (correct) solutions of the underlying task [5,6]: If the students requires a hint on how to change her attempt to get closer to a correct solution, it can be compared to a similar example from the set, and the dissimilarities between her attempt and the example can be contrasted or highlighted in order to help the student to improve her own solution [7,8,9].…”

Designing a Knowledge Representation Approach for the Generation of Pedagogical Interventions by MTTs

“…Finally, the GIL tutoring system (Reiser et al 1992) is able to generate interventions for LISP programming tasks, using a production system specially extended for that purpose. Stamper et al (2013) also worked on the problem of generating next-step hints within a logic proof tutor. Their approach consists in generating Markov Decision Processes (MDPs) to graph the states transitions in a task case using traces of previous learners and experts.…”

“…This approach greatly differs from Astus's. Whereas Astus uses task-independent processes to generate pedagogical content from an expert defined model of the task, Stamper et al (2013) use a task-independent process to generate MDPs that are used to instantiate task-specific hints.…”