Development and Performance Assessment of a Digital Serious Game to Assess Multi-Patient Care Skills in a Simulated Pediatric Emergency Department

Abstract

Objective. Multi-patient care is important among medical trainees in an emergency department (ED). While resident efficiency is a typically measured metric, multi-patient care involves both efficiency and diagnostic / treatment accuracy. Multi-patient care ability is difficult to assess, though simulation is a potential alternative. Our objective was to generate validity evidence for a serious game in assessing multi-patient care skills among a variety of learners. Methods. This was a cross-sectional validation study using a digital serious game VitalSignsTM simulating multi-patient care within a pediatric ED. Subjects completed 5 virtual “shifts,” triaging, stabilizing, and discharging or admitting patients within a fixed time period; patients arrived at cascading intervals with pre-programmed deterioration if neglected. Predictor variables included generic multi-tasking ability, video game experience, medical knowledge, and clinical efficiency with real patients. Outcome metrics in 3 domains measured diagnostic accuracy (i.e. critical orders, diagnoses), efficiency (i.e. number of patients, time-to-order) and critical thinking (number of differential diagnoses); MANOVA determined differences between novice learners and expected expert physicians. Spearman Rank correlation determined associations between levels of expertise. Results. Ninety-five subjects’ gameplays were analyzed. Diagnostic accuracy and efficiency distinguished skill level between residency trained (residents, fellows and attendings) and pre-residency trained (medical students and undergraduate) subjects, particularly for critical orders, patients seen, and correct diagnoses (p < 0.003). There were moderate to strong correlations between the game’s diagnostic accuracy and efficiency metrics compared to level of training, including patients seen (rho = 0.47, p < 0.001); critical orders (rho = 0.80, p < 0.001); time-to-order (rho = −0.24, p = 0.025); and correct diagnoses (rho = 0.69, p < 0.001). Video game experience also correlated with patients seen (rho = 0.24, p = 0.003). Conclusion. A digital serious game depicting a busy virtual ED can distinguish between expected experts in multi-patient care at the pre- vs. post-residency level. Further study can focus on whether the game appropriately assesses skill acquisition during residency.

The Association Between Multitasking and Multi-Patient Care Skills in a Simulated Patient Care Video Game Among Second Year Medical Students Based on Specialty Choice

“…This was a planned cross-sectional sub-analysis within a parent study that provided validity evidence on a serious game called VitalSigns:ED, a real-time simulation for multi-patient care (Luu et al, 2020). Our subanalysis focused on a second year medical students (MS-IIs) at a single 4-year allopathic medical school with a pre-clinical curriculum lasting 2 years.…”

“…We used a serious game platform to simulate a multi-patient care environment with evolving and deteriorating emergency department (ED) patients (http://www.breakawaygames.com/VitalSigns/). This was a desktop computer serious game using a keyboard and mouse, which was piloted among pediatric specialists (Luu et al, 2020) and also validated in an adult ED version (Figure 2) (Mohan et al, 2014a, 2014b, 2015). Within the game, multiple patients with evolving physiologies require re-assessment, prioritization of orders, and real-time diagnostic tests and treatments between multiple patients; a differential diagnosis was requested upon admission or discharge of the patient.…”

Feeling the flow with a serious game workshop: GridlockED as Medical Education 2 study (GAME2 study)

“…Similarly, some of our contemporaries in the pediatric EM world have been working on a simulation game-something that is more immersive, perhaps, and focuses much more on the clinical decisionmaking processes within a multipatient environment. 19 Whereas…”

“…GridlockED emphasizes systems-level thinking and provides an overview of an entire department, the gaming computer-based simulation by Luu and colleagues 19 focuses more on helping junior trainees jump between multiple active cases. Either method may ultimately prove to be useful, but more clarification studies are required to better understand how to teach and learn the skill of task switching.…”

Gaming in the Time of COVID-19

“…Luu, Talbot, Fung, Ben-Isaac, Espinoza, Fischer, Cho, Sargsyan, Korand and Chang (2020) demonstrate the capabilities of medical digital simulation games for a different purpose: the assessment of multi-patient care skills. They investigate a game with a simulated pediatric emergency department for performance assessment.…”