Design and usability assessment of a dialogue‐based cognitive tutoring system to model expert problem solving in research design

Abstract

Technology provides the means to create useful learning and practice environments for learners. Well‐designed cognitive tutor systems, for example, can provide appropriate learning environments that feature cognitive supports (ie, scaffolding) for students to increase their procedural knowledge. The purpose of this study was to conduct a series of usability tests of a dialogue‐based design framework for the presentation of domain knowledge and assess how it can be used to actively engage learners in learning about research methods. Three formal usability assessments and an instructor adoption assessment were conducted during the development of the tutoring system. Each usability assessment employed diverse data collection methods to ensure broad and in‐depth coverage of findings. The findings revealed that the dialogue metaphor enabled natural and participatory interactions between the system and users. The feedback prompts or hints and support resources provided opportunities for learning during the process of problem solving. Future research to extend the support of usability assessments is also discussed.

Technology and Feedback Design

“…However, large class sizes, coupled with increasing time pressures and staffing costs, can limit the ability of educators to provide this personal support (Chu, Yang, Tseng, & Yang, 2014;W.-C. Hung, Smith, & Smith, 2015). As computer technology develops, intelligent tutoring systems have emerged as a means of providing students with interactive, flexible, and focused personal learning support (W.-C. Hung et al, 2015;Steif, Fu, & Kara, 2016). Such support is particularly valuable as one-to-one tutoring from an educator has been shown to improve student achievement (Chu et al, 2014).…”

“…Intelligent tutoring systems may guide students through a learning exercise or seek to diagnose learning difficulties and provide corrective feedback (Chu et al, 2014;W.-C. Hung et al, 2015). While intelligent tutoring systems may be designed around a number of systems, recent research regarding intelligent tutoring systems has focused on cognitive tutoring (W.-C. Hung et al, 2015;Steif et al, 2016).…”

“…Intelligent tutoring systems may guide students through a learning exercise or seek to diagnose learning difficulties and provide corrective feedback (Chu et al, 2014;W.-C. Hung et al, 2015). While intelligent tutoring systems may be designed around a number of systems, recent research regarding intelligent tutoring systems has focused on cognitive tutoring (W.-C. Hung et al, 2015;Steif et al, 2016). Cognitive tutoring mechanisms use a model of cognitive behaviour to interpret and evaluate student learning behaviours which take place within the tutoring system, typically centring on a problem-solving exercise (Chu et al, 2014;W.-C. Hung et al, 2015).…”

“…While intelligent tutoring systems may be designed around a number of systems, recent research regarding intelligent tutoring systems has focused on cognitive tutoring (W.-C. Hung et al, 2015;Steif et al, 2016). Cognitive tutoring mechanisms use a model of cognitive behaviour to interpret and evaluate student learning behaviours which take place within the tutoring system, typically centring on a problem-solving exercise (Chu et al, 2014;W.-C. Hung et al, 2015). However, a significant criticism of cognitive tutoring systems has emerged from disciplines such as engineering and mathematics, which require students to undertake problem-solving tasks (Chu et al, 2014;Steif et al, 2016).…”

“…The design of cognitive tutoring systems varies significantly between disciplines. Students may complete a series of calculations in a mathematics or engineering context, or work through a set of interactive, problem-based scenarios (Chu et al, 2014;W.-C. Hung et al, 2015;Steif et al, 2016). Cognitive tutoring systems typically provide students with immediate feedback, either when students submit an answer or at a series of preselected points in the system (Chu et al, 2014;Steif et al, 2016).…”