Virtual Standardized Patient Simulation

Abstract

Introduction High-value care (HVC) suggests that good history taking and physical examination should lead to risk stratification that drives the use or withholding of diagnostic testing. This study describes the development of a series of virtual standardized patient (VSP) cases and provides preliminary evidence that supports their ability to provide experiential learning in HVC. Methods This pilot study used VSPs, or natural language processing–based patient avatars, within the USC Standard Patient platform. Faculty consensus was used to develop the cases, including the optimal diagnostic testing strategies, treatment options, and scored content areas. First-year resident physician learners experienced two 90-minute didactic sessions before completing the cases in a computer laboratory, using typed text to interview the avatar for history taking, then completing physical examination, differential diagnosis, diagnostic testing, and treatment modules for each case. Learners chose a primary and 2 alternative “possible” diagnoses from a list of 6 to 7 choices, diagnostic testing options from an extensive list, and treatments from a brief list ranging from 6 to 9 choices. For the history-taking module, both faculty and the platform scored the learners, and faculty assessed the appropriateness of avatar responses. Four randomly selected learner-avatar interview transcripts for each case were double rated by faculty for interrater reliability calculations. Intraclass correlations were calculated for interrater reliability, and Spearman ρ was used to determine the correlation between the platform and faculty ranking of learners' history-taking scores. Results Eight VSP cases were experienced by 14 learners. Investigators reviewed 112 transcripts (4646 learner query-avatar responses). Interrater reliability means were 0.87 for learner query scoring and 0.83 for avatar response. Mean learner success for history taking was scored by the faculty at 57% and by the platform at 51% (ρ correlation of learner rankings = 0.80, P = 0.02). The mean avatar appropriate response rate was 85.6% for all cases. Learners chose the correct diagnosis within their 3 choices 82% of the time, ordered a median (interquartile range) of 2 (2) unnecessary tests and completed 56% of optimal treatments. Conclusions Our avatar appropriate response rate was similar to past work using similar platforms. The simulations give detailed insights into the thoroughness of learner history taking and testing choices and with further refinement should support learning in HVC.

The Use of Virtual Standardized Patients for Practice in High Value Care

“…We did not formally calculate the ability of a finding to discriminate among DDx items but informally agreed that “must ask” items were akin to “clinically discriminating” and “nice to ask” items were akin to “thoroughness” items as described by Yudkowsky et al 37 The end of case summary shared with the learner also contained text that explained the diagnosis and why testing may or may not have been indicated. For more details on how the cases were created, consensus scoring developed, and pilot testing for avatar response rate, please see the study group's prior work 38 …”

Next-Generation Simulation—Integrating Extended Reality Technology Into Medical Education

“…The increased interactivity afforded by XR also works to further deepen learner immersion and satisfaction with the virtual experience, aiding in active learning and increasing learner motivation (Hariri et al, 2004;Stepan et al, 2017;Hudson et al, 2019). Due to the virtual nature of XR, educational experiences can be practiced repeatedly and learners can be objectively assessed through the virtual platform (Dubin et al, 2017;Bond et al, 2019;Maicher et al, 2019;Pottle, 2019). Thanks to their portable and relatively inexpensive form, new commercial XR devices can also increase accessibility to educational experiences outside of the classroom (Moro et al, 2017;Kourtesis et al, 2019).…”

“…Recent XR innovations in motion tracking allow for natural human movements in the VE that help students practice body language and other nonverbal communication techniques (Kourtesis et al, 2020;Aseeri and Interrante, 2021). Innovative medical schools and affiliated hospitals are using a variety of VSP platforms constructed in-house or using proprietary technology developed with industry partners (Maicher et al, 2017;McGrath et al, 2018;Bond et al, 2019;Guetterman et al, 2019) (Figure 4).…”

“…Research has shown that regular practice with VSPs can improve medical student proficiency in empathetic communication, discussing sensitive health information, and conflict resolution (Courteille et al, 2014;Kron et al, 2017;Fertleman et al, 2018). Several studies have demonstrated that different VSP platforms are effective in testing student ability to come up with correct differential diagnoses based on a VSP presentation, with about 80% of learners choosing the correct primary and secondary possible diagnoses (Maicher et al, 2017;Bond et al, 2019). VSP simulators with XR UIs can realistically test the clinical reasoning and decision-making skills of nursing and medical students in response to emergency scenarios such as anaphylactic shock, heart attack, and cardiac arrest without putting patients or students at risk (McGrath et al, 2018;Hauze et al, 2019;Pottle, 2019;Rushton et al, 2020).…”

“…Aside from testing diagnostic and crisis-response skills, another advantage of VSP platforms is the built-in objective assessment of student communication and diagnostic skills based on learner responses in the VE. Studies have compared the automated grading ability of VSP platforms to traditional grading by a human instructor and found them to be largely equivalent in accuracy of feedback, and superior in speed of feedback due to the electronic nature of the platforms (Bond et al, 2019;Maicher et al, 2019). Additionally, the virtual nature of VSP platforms allow for repeatable practice with variable scenarios outside of the confines of SP availability and classroom space (Pottle, 2019).…”