Computer-Based Case Simulations for Assessment in Health Care: A Literature Review of Validity Evidence

Abstract

Simulation for education and training in health-care professions has been widely applied. However, its value as an assessment tool for competence is not fully known. Logistical barriers of simulation-based assessments have led some health-care organizations to utilize computer-based case simulations (CCSs) for assessment. This article provides a review of the literature on the identification of psychometrically sound, CCS instruments designed to measure decision-making competence in health-care professionals. CINAHL, MEDLINE, and Ovid databases identified 84 potentially relevant articles published between January 2000 and May 2017. A total of 12 articles met criteria for inclusion in this review. Findings of these 12 articles indicate that summative assessment in health care using CCSs in the form of clinical scenarios is utilized to assess higher order performance aspects of competence in the form of decision-making. Psychometric strength was validated in eight articles and supported by four replication studies. Two of the eight articles reported evidence of construct validity and support the need for evidence based on a theoretical framework. This literature review offers implications for further research on the use of CCS tools as a method for assessment of competence in health-care professionals and the need for psychometric evidence to support it.

Using the RETAIN Tabletop Simulator as a Summative Assessment Tool for Neonatal Resuscitation Healthcare Professionals: A Pilot Study

“…Poorly designed feedback in some summative assessment methods detracts from motivation and depth of learning (8). Methods that focus on clinical practice assessment such as simulations are, therefore, preferred (10,24).…”

“…Simulation-based clinical scenarios are the best method to demonstrate and measure HCPs' learning (10). While simulation is frequently used in training, it remains an underreported approach to assess HCPs' ongoing competency, such as during recertification (10). Barriers to simulation use include the resource-heavy requirements of a high-fidelity manikin, simulation lab, and trained instructor.…”

“…Also, simulation can be logistically challenging, as multiple learners and instructors must take time off from work to attend the scheduled sessions. Serious games like RETAIN may address these barriers (9,10).…”

“…Summative assessments evaluate learners by measuring their individual performance on a particular task at the end of an instructional unit (8). While simulation-based summative assessment demonstrates competence, resource constraints hinder its widespread uptake (9,10). Recognizing these barriers, alternative methods like simulation-based serious games may be an alternative (10).…”

“…While simulation-based summative assessment demonstrates competence, resource constraints hinder its widespread uptake ( 9 , 10 ). Recognizing these barriers, alternative methods like simulation-based serious games may be an alternative ( 10 ). Serious games incorporate a challenging goal and scoring system to educate players on useful knowledge or skills, thereby creating an interactive and motivating problem-based learning experience ( 11 ).…”