Linking Performance Measures and Measurement Purpose in Scenario-Based Training

Abstract

Despite its importance, there are scant guidelines to assist instructional development experts with the scenario-based training process. In particular, little information exists that links different types of human performance measures with the type of performance assessed and the purpose of training. In an effort to provide new insights on this important issue, the present study examined the appropriateness of four rating formats (i.e., checklists, frequency count, distance and discrepancy, and rating scale) for six performance outcomes, three process variables, and three purposes of scenario-based training. Expert judgments were collected via a short survey. The results indicated that for nine of the 12 variables a single measurement strategy was most appropriate. However, the results also indicated that experts tended to over rely on rating scales as their preferred choice. Implications for practice and suggested requirements for future research are provided.

Storytelling as an Instructional Method: Definitions and Research Questions

“…Literature on scenario-based instruction and training refer often to many of the theoretical publications listed under case-based instruction above. However, the method is extended to include the training of teams (Cooke, 2002;DuBois & Gillan, 2000;Oser et al, 1999;Salas et al, 2001;Van Berlo, 2005) and strives to develop highly elaborate measurement structures (Baker et al, 2004;Spiker et al, 2006; for a review of the levels of evaluation see Kirkpatrick, 1987). Completing tasks, real or synthetic, are central to the design and objectives of scenarios (Cooke, in press).…”

“…In scenario-based instruction the problem is characterized by fixed solution criteria and the learner is positioned in an interactive, real-time experience that allows for a variety of solution paths (Salas, Wilson, Priest, & Guthrie, 2006). Scenarios are constructed using information from cases or instructor experience and are creatively authored to measure specific performance outcomes (Baker, Kuang, Feinberg, & Radtke, 2004). Records of individual and team trainee responses can be used in a sto-…”