Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring

Abstract

SimStudent is a machine-learning agent initially developed to help novice authors to create cognitive tutors without heavy programming. Integrated into an existing suite of software tools called Cognitive Tutor Authoring Tools (CTAT), SimStudent helps authors to create an expert model for a cognitive tutor by tutoring SimStudent on how to solve problems. There are two different ways to author an expert model with SimStudent. In the context of Authoring by Tutoring, the author interactively tutors SimStudent by posing problems to SimStudent, providing feedback on the steps performed by SimStudent, and also demonstrating steps as a response to SimStudent's hint requests when SimStudent cannot perform steps correctly. In the context of Authoring by Demonstration, the author demonstrates solution steps, and SimStudent attempts to induce underlying domain principles by generalizing those worked-out examples. We conducted evaluation studies to investigate which authoring strategy better facilitates authoring and found two key results. First, the expert model generated with Authoring by Tutoring is better and has higher accuracy while maintaining the same level of completeness than the one generated with Authoring by Demonstration. The reason for this better accuracy is that the expert model generated by tutoring benefits from negative feedback provided for SimStudent's incorrect production applications. Second, authoring by Tutoring requires less time than Authoring by Demonstration. This enhanced authoring efficiency is partially because (a) when Authoring by Demonstration, the author needs to test the quality of the expert model, whereas the formative assessment of the expert model is done naturally by

Evaluation Methods for Intelligent Tutoring Systems Revisited

“…A good survey of the prospects for use of simulated learners in evaluating systems can be found in Vanlehn et al (1994). More recent work by Matsuda et al (2014) provides a good example of system evaluation with simulated learners. Simulated learners can help with formative assessments of learning environments or in validating specific components of a system, but testing with simulated learners cannot replace full summative evaluation and studies of educational impact.…”

Fuzzy comprehensive evaluation model of interuniversity collaborative learning based on network

“…Y. Liu [11] analyzed educational evaluation results based on decision tree algorithm. H. Xiaorong [12] presented networkcourse learning e ect evaluation model based on threelayer Bayesian network. A web-based framework for usercentred evaluation of end-user experience in adaptive and personalized E-Learning systems was proposed by C. Mulwa [13].…”

Inovation in Education – Fundamental Request of Knowledge Society

“…The process is more complex and involves the transformation of values that have to be provided, the information that have to be taught, the methods that have to be used in educational activities etc. Of course, innovation in education must take place simultaneously in all these components, any imbalance may compromise the idea of "educational curriculum" and may affect academic achievement of students (N. Matsuda and W. Cohen, 2014). Therefore, for a better understanding of innovation in education' significance, in this study we will focus on its main indicators: informational content, teaching materials and teaching / evaluation methods.…”