Adaptive simulation training using cumulative sum: a randomized prospective trial

Hu, Yinin; Brooks, Kendall D.; Kim, Helen; Mahmutovic, Adela; Choi, Joanna; Le, Ivy A.; Kane, Bartholomew J.; McGahren, Eugene D.; Rasmussen, Sara K.

doi:10.1016/j.amjsurg.2015.08.030

Cited by 11 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proficiency did not depend on initial training method, which supports continued exploration of CuSum for real-time monitoring and objective feedback to guide the training of technical skills, as the CuSum method demonstrated a significant reduction in time required to learn complex tasks. 8 Specific to medical students, these findings suggest a value in training students before graduation in certain technical skills needed during internship through a longitudinal simulation curriculum throughout medical school education vs targeted clinical electives such as surgery boot camp or surgical clerkships. 12 …”

Section: Discussionmentioning

confidence: 97%

“…8 In brief, preclinical medical students were randomized into CuSum-based vs volume-based arms. Participants underwent simulation training on suturing, intubation, and CVC skills.…”

Section: Methodsmentioning

confidence: 99%

“…Performance was assessed against a checklist of a priori objective measures for each procedural task and scored via expert evaluation. 8 …”

Section: Methodsmentioning

confidence: 99%

“…Additionally, high rates of technical proficiency were attained across the various procedural arms in both study arms. 8 …”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

Ramirez

Kim

et al. 2018

Journal of Surgical Education

Self Cite

View full text Add to dashboard Cite

OBJECTIVE Cumulative sum (CuSum) is a real-time proficiency-monitoring tool adapted for simulation-based training. This study’s objective was to investigate long-term outcomes of a double blinded, randomized control trial conducted with medical students assessing CuSum-guided curriculum against volume-based standards. The trial found a nearly 20% reduction in practice time to reach proficiency using the CuSum curriculum but long-term effects of decreased practice volume on proficiency is unknown. DESIGN Prior participants completed a survey assessing confidence, exposure, and feedback at 12 to 18 months following trial completion. They underwent retention testing of suturing, intubation, and central venous catheter placement (CVC), which was video-recorded and assessed by an expert evaluator. Baseline characteristics among repeat subjects were compared using chi-squared tests. Retention and initial trial outcome were compared using paired parametric statistical methods. SETTING The study was conducted at a major tertiary care center and training hospital. PARTICIPANTS Medical students, which completed the initial randomized control trial were eligible for enrollment. A total of 30/46(65%) responded to the survey, whereas 33/46(72%) completed retention testing. RESULTS Average scores and decay in procedural tasks over time for suturing, intubation and CVC were 91.6% (−4.7%), 86.1% (−4.1%), and 76.2% (−14.8%), respectively. Compared to the control group, the CuSum group mean difference in retention evaluation scores was −5.6% (p = 0.12). Confidence was not associated with initial or retention testing performance in any procedural task. Higher confidence was associated with additional exposure to the procedural task in suturing and intubation (p = 0.03 and p = 0.02, respectively). For intubation, higher confidence was reported by participants who received positive feedback (p = 0.01), and those assigned to the volume-based training arm (p = 0.03). CONCLUSION CuSum-guided training was equivalent to conventional training for suturing, intubation, and CVC. These findings importantly suggest medical students can retain competency in invasive surgical tasks with modest decay in proficiency over time regardless of initial training method.

show abstract

Section: Discussionmentioning

confidence: 97%

“…8 In brief, preclinical medical students were randomized into CuSum-based vs volume-based arms. Participants underwent simulation training on suturing, intubation, and CVC skills.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

Ramirez

Kim

et al. 2018

Journal of Surgical Education

Self Cite

View full text Add to dashboard Cite

show abstract

“…A specific type of eLearning-competency-based eLearning-has shown promising results in medical education with decreased overall learning time and improved accuracy. [17][18][19] Competency-based eLearning requires the user to demonstrate knowledge, or skill proficiency, through an assessment prior to advancement. Hu et al 17 compared a competencybased eLearning simulation and traditional practice of surgical skills with first-and second-year medical students (n = 48).…”

Section: Neuroanatomy Elearningmentioning

confidence: 99%

A Qualitative Examination Detailing Medical Student Experiences of a Novel Competency-Based Neuroanatomy eLearning Intervention Designed to Bridge a Gap Within an Integrated Medical Curriculum

Hart

Brueckner‐Collins

Bergden

2021

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

Background: A 1-year time-gap between first- and second-year neuroanatomy courses was created at our institution as a result of restructuring the curriculum from systems-based to an integrated format. Additionally, neuroanatomy hours decreased significantly (48.8%) when transitioning to an integrated curriculum, similar to other medical schools. Competency-based eLearning in medical education has shown promising results with decreasing overall learning time and improving accuracy. To date, competency-based eLearning has not yet been explored in neuroanatomy education. Objective: The purpose of this study is to develop and assess a novel competency-based neuroanatomy eLearning intervention for second-year medical students designed to bridge a 1-year time-gap, without adding significant instructional hours, in an integrated curriculum. Methods: A competency-based eLearning intervention encompassing the major tracts, brainstem anatomy, and an interactive case featuring a simulated patient experience was developed in the Articulate Storyline® platform. Student usage data, single-session course evaluations, and a focus group were used to evaluate the module’s effectiveness. Results: Student usage data showed an average completion time of M = 2:59:25 hours which fit within the scheduled 3-hour timeframe. Students rated the module’s overall effectiveness as M = 3.65 (out of 4) on a single-session evaluation. A focus group provided qualitative feedback suggesting improvements to the eLearning module in the domains of content, mechanics, and timing. Conclusion: A competency-based neuroanatomy eLearning intervention shows promising initial results to bridge a 1-year educational gap within an integrated curriculum. Overall, students described this educational tool as helpful and outlined ways in which to improve this resource.

show abstract

Ensemble deep learning for the prediction of proficiency at a virtual simulator for robot-assisted surgery

et al. 2022

View full text Add to dashboard Cite

Background Artificial intelligence (AI) has the potential to enhance patient safety in surgery, and all its aspects, including education and training, will derive considerable benefit from AI. In the present study, deep-learning models were used to predict the rates of proficiency acquisition in robot-assisted surgery (RAS), thereby providing surgical programs directors information on the levels of the innate ability of trainees to facilitate the implementation of flexible personalized training. Methods 176 medical students, without prior experience with surgical simulators, were trained to reach proficiency in five tasks on a virtual simulator for RAS. Ensemble deep neural networks (DNN) models were developed and compared with other ensemble AI algorithms, i.e., random forests and gradient boosted regression trees (GBRT). Results DNN models achieved a higher accuracy than random forests and GBRT in predicting time to proficiency, 0.84 vs. 0.70 and 0.77, respectively (Peg board 2), 0.83 vs. 0.79 and 0.78 (Ring walk 2), 0.81 vs 0.81 and 0.80 (Match board 1), 0.79 vs. 0.75 and 0.71 (Ring and rail 2), and 0.87 vs. 0.86 and 0.84 (Thread the rings 2). Ensemble DNN models outperformed random forests and GBRT in predicting number of attempts to proficiency, with an accuracy of 0.87 vs. 0.86 and 0.83, respectively (Peg board 2), 0.89 vs. 0.88 and 0.89 (Ring walk 2), 0.91 vs. 0.89 and 0.89 (Match board 1), 0.89 vs. 0.87 and 0.83 (Ring and rail 2), and 0.96 vs. 0.94 and 0.94 (Thread the rings 2). Conclusions Ensemble DNN models can identify at an early stage the acquisition rates of surgical technical proficiency of trainees and identify those struggling to reach the required expected proficiency level.

show abstract

Adaptive simulation training using cumulative sum: a randomized prospective trial

Cited by 11 publications

References 24 publications

Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

A Qualitative Examination Detailing Medical Student Experiences of a Novel Competency-Based Neuroanatomy eLearning Intervention Designed to Bridge a Gap Within an Integrated Medical Curriculum

Ensemble deep learning for the prediction of proficiency at a virtual simulator for robot-assisted surgery

Contact Info

Product

Resources

About