When Are Trainees Ready to Perform Transvaginal Ultrasound? An Observational Study

Abstract

Purpose The purpose of this study was two-fold: (1) To determine how the number of ultrasound scans a trainee has completed predicts the trainee’s diagnostic accuracy when performing transvaginal ultrasound examinations, and (2) to examine the utility of simulation-based assessment of ultrasound competence for determining readiness for independently performing examinations. Materials and Methods 101 OB/GYN trainees were surveyed regarding their clinical experience and the number of scans they had completed. All participants completed five different cases on a transvaginal virtual-reality ultrasound simulator (Scantrainer, Medaphor). The participants’ diagnostic accuracy was recorded and expert raters evaluated their performance using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. The utility of simulation-based assessments was assessed with respect to reliability, validity, acceptability, and costs. The main outcome was diagnostic accuracy for five different ultrasound cases. Results Although the number of scans was associated with diagnostic accuracy (p = 0.006), it was a poor predictor (AUC 0.69) of diagnostic accuracy. Only 56.6 % (n = 34) of participants who had more than 100 transvaginal scans demonstrated a diagnostic accuracy of 0.80 or above. The reliability of the OSAUS assessments was high (ICC 0.82) and the majority of participants supported the use of simulation-based assessments for future licensing exams (70.3 %). The running costs of simulation-based assessments (154 EUR per participant) were lower than for practical examinations using real patients. Conclusion The number of completed ultrasound scans was a poor predictor of the trainees’ diagnostic accuracy. Instead, simulation-based assessments can be used to ensure that trainees are ready for independently performing future scans.

Multicenter randomized trial exploring effects of simulation‐based ultrasound training on obstetricians' diagnostic accuracy: value for experienced operators

“…First, the simulation-based training provided in this study is a very brief intervention relative to the presumably large volume of scans completed by the experienced obstetricians. Our findings, therefore, give further credence to the concern raised in recent studies regarding the reliability of volume of scans as a measure of competence in ultrasound 3,5 . Second, according to some learning theories 13 , structured teaching, such as simulation-based ultrasound training, may have negative effects for experienced obstetricians, who already have inveterate habits in performing obstetric ultrasound 13 .…”

“…The learning curve is long, and it may be difficult to reach and maintain mastery if training relies on clinical practice alone. In fact, a number of studies have shown that clinical experience does not necessarily ensure competence, which may be explained by a lack of initial systematic training, as well as skill decay over time. Lack of ultrasound competence can cause diagnostic errors that lead to inappropriate obstetric management, which may subsequently have serious implications for patient care and outcomes.…”

Simulation-Based Ultrasound Training in Obstetrics and Gynecology: A Systematic Review and Meta-Analysis

“…In addition to the large time resources required for clinical training, ultrasound examinations can be accompanied by some discomfort for the patient when the ultrasound operator is inexperienced [5]. Insufficient initial training may lead to failure to achieve expert levels of diagnostic certainty regardless of the number of completed scans [6]. For these reasons, there has been an increased focus on more effective methods for ultrasound training such as simulation-based medical education, which in other fields of practice has been shown to result in large improvements in trainee skills and moderate improvements in patient outcomes [7].…”

Quantitating skill acquisition with optical ultrasound simulation

“…5 Despite this, a recent study showed that, while there is an association between number of scans performed and diagnostic accuracy, the number of scans performed is not a sufficiently robust predictor of accuracy to ensure proficiency. 6 As a result of both the time and subjectivity involved in developing ultrasound skills, there is evolving interest in the use of ultrasound simulation training to enable practical skills to be developed at a time and place convenient to trainee and trainer and as an objective and standardised assessment of competency. [6][7][8] Several platforms and systems have been described and are reported to lead to sustained improvements in performance, which can be reproduced in the clinical environment.…”

“…6 As a result of both the time and subjectivity involved in developing ultrasound skills, there is evolving interest in the use of ultrasound simulation training to enable practical skills to be developed at a time and place convenient to trainee and trainer and as an objective and standardised assessment of competency. [6][7][8] Several platforms and systems have been described and are reported to lead to sustained improvements in performance, which can be reproduced in the clinical environment. [7][8][9][10][11][12]…”