Assessment of performance measures and learning curves for use of a virtual-reality ultrasound simulator in transvaginal ultrasound examination

Madsen, Mette; Konge, Lars; Nørgaard, Lone Nikoline; Tabor, Ann; Ringsted, Charlotte; Klemmensen, A.; Ottesen, Bent; Tolsgaard, Martin G.

doi:10.1002/uog.13400

Cited by 101 publications

(112 citation statements)

References 20 publications

Supporting

Mentioning

107

Contrasting

Unclassified

Order By: Relevance

“…Future studies could assess the time and number of attempts the novices need to reach the pass/fail level and to compare this to the number of attempts and time needed to reach the learning curve plateau. This will highlight whether novices acquire additional learning beyond the pass/fail level, as previous studies have suggested [33, 34]. …”

Section: Discussionmentioning

confidence: 88%

“…Previous studies show that there is a difference between pass/fail level and learning curve plateau, indicating that the trainees might be proficiently trained when reaching the pass/fail level, but still have potential to reach a higher level of competence until reaching a learning curve plateau [30, 33, 34]. When reaching the learning curve plateau the trainees would not benefit from further simulation training.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Ensuring Basic Competence in Thoracentesis

et al. 2019

View full text Add to dashboard Cite

Background: Trocar pigtail catheter thoracentesis (TPCT) is a common procedure often performed by junior physicians. Simulation-based training may effectively train physicians in the procedure prior to performing it on patients. An assessment tool with solid validity evidence is necessary to ensure sufficient procedural competence. Objectives: Our study objectives were (1) to collect evidence of validity for a newly developed pigtail catheter assessment tool (Thoracentesis Assessment Tool [ThorAT]) developed for the evaluation of TPCT performance and (2) to establish a pass/fail score for summative assessment. Methods: We assessed the validity evidence for the ThorAT using the recommended framework for validity by Messick. Thirty-four participants completed two consecutive procedures and their performance was assessed by two blinded, independent raters using the ThorAT. We compared performance scores to test whether the assessment tool was able to discern between the two groups, and a pass/fail score was established. Results: The assessment tool was able to discriminate between the two groups in terms of competence level. Experienced physicians received significantly higher test scores than novices in both the first and second procedure. A pass/fail score of 25.2 points was established, resulting in 4 (17%) passing novices and 1 (9%) failing experienced participant in the first procedure. In the second procedure 9 (39%) novices passed and 2 (18%) experienced participants failed. Conclusions: This study provides a tool for summative assessment of competence in TPCT. Strong validity evidence was gathered from five sources of evidence. A simulation-based training program using the ThorAT could ensure competence before performing thoracentesis on patients.

show abstract

Section: Discussionmentioning

confidence: 88%

Section: Discussionmentioning

confidence: 99%

Ensuring Basic Competence in Thoracentesis

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The autonomous stage also proved individuals to have a greater ability to detect errors and improved anticipation and decision-making, and therefore minimal variations and errors [37]. Several studies on the evaluation of ultrasound skills support this theory [38, 39]. …”

Section: Discussionmentioning

confidence: 94%

Developing and Gathering Validity Evidence for a Simulation-Based Test of Competencies in Lung Ultrasound

et al. 2018

View full text Add to dashboard Cite

Background: Clinical lung ultrasound (LUS) is a fast bedside diagnostic tool which can assist clinicians in decisions regarding the treatment and monitoring of patients with respiratory symptoms. LUS training and education differ widely, and is often done in a clinical setting, with potential risks for patients if decisions are made based on the wrong interpretations. No clear guidelines or recommendations for objective and standardized assessment of LUS skills exist, and those that do are often based on a fixed time-frame or an arbitrary number of examinations performed; this does not ensure adequate competencies. Objectives: The study aimed to develop and gather validity evidence for a practical, simulation-based test in LUS. Methods: Nine cases were developed in collaboration with 3D Systems Healthcare, Littleton, CO, USA, representing the most common diagnosis and sonographic findings in patients with respiratory symptoms. Thirty-six participants with different levels of competence in LUS, completed the test. The participants were divided into groups, i.e., novices, intermediates, and experienced, according to their experience with LUS, the number of examinations they had performed, and any research they had conducted. Their answers were used for item analyses. Results: The intraclass correlation coefficient, Cronbachs’ α, was 0.69 summarized, and there was a statistically significant difference (p < 0.001) between the novices and the trained participants (intermediates and experienced). A pass/fail score of 16 points was calculated according to the contrasting-groups method. Conclusion: We developed a test for the assessment of clinical competencies in LUS. The test proved solid validity evidence, and a pass/fail standard without any false-negatives, and only 2 explained false-positives.

show abstract

“…Finally, criterion‐based pass/fail standards have been established for the OSAUS scale in a previous study 30. The validity evidence of simulator metrics used in the assessment of simulated performances has been examined in a previous study 32. Only metrics that demonstrated construct validity were included in the analyses of the present study.…”

Section: Methodsmentioning

confidence: 99%

The effect of dyad versus individual simulation‐based ultrasound training on skills transfer

et al. 2015

View full text Add to dashboard Cite

ContextDyad practice may be as effective as individual practice during clinical skills training, improve students’ confidence, and reduce costs of training. However, there is little evidence that dyad training is non‐inferior to single‐student practice in terms of skills transfer.ObjectivesThis study was conducted to compare the effectiveness of simulation‐based ultrasound training in pairs (dyad practice) with that of training alone (single‐student practice) on skills transfer.MethodsIn a non‐inferiority trial, 30 ultrasound novices were randomised to dyad (n = 16) or single‐student (n = 14) practice. All participants completed a 2‐hour training programme on a transvaginal ultrasound simulator. Participants in the dyad group practised together and took turns as the active practitioner, whereas participants in the single group practised alone. Performance improvements were evaluated through pre‐, post‐ and transfer tests. The transfer test involved the assessment of a transvaginal ultrasound scan by one of two clinicians using the Objective Structured Assessment of Ultrasound Skills (OSAUS).ResultsThirty participants completed the simulation‐based training and 24 of these completed the transfer test. Dyad training was found to be non‐inferior to single‐student training: transfer test OSAUS scores were significantly higher than the pre‐specified non‐inferiority margin (delta score 7.8%, 95% confidence interval −3.8–19.6%; p = 0.04). More dyad (71.4%) than single (30.0%) trainees achieved OSAUS scores above a pre‐established pass/fail level in the transfer test (p = 0.05). There were significant differences in performance scores before and after training in both groups (pre‐ versus post‐test, p < 0.01) with large effect sizes (Cohen's d = 3.85) and no significant interactions between training type and performance (p = 0.59). The dyad group demonstrated higher training efficiency in terms of simulator score per number of attempts compared with the single‐student group (p = 0.03).ConclusionDyad practice improves the efficiency of simulation‐based training and is non‐inferior to individual practice in terms of skills transfer.

show abstract

Assessment of performance measures and learning curves for use of a virtual-reality ultrasound simulator in transvaginal ultrasound examination

Abstract: Objective

Cited by 101 publications

References 20 publications

Ensuring Basic Competence in Thoracentesis

Ensuring Basic Competence in Thoracentesis

Developing and Gathering Validity Evidence for a Simulation-Based Test of Competencies in Lung Ultrasound

The effect of dyad versus individual simulation‐based ultrasound training on skills transfer

Contact Info

Product

Resources

About