Evaluation of tools to assess operative competence in endovascular procedures: a systematic review

Abstract

Background: With an increase in the use of endovascular interventions as an alternative to open surgery and the unique technical skills required, current methods for assessing the competence of vascular surgery trainees may not be optimal, suggesting a need for a shift in assessment modalities. We conducted this systematic review to explore current assessment methods used in vascular surgery training to assess competence specific to endovascular procedures. Methods: A comprehensive literature search was performed with a structured search strategy using terms focusing on endovascular procedures and assessment. Inclusion and exclusion criteria were used in order to screen for suitable articles. Results: We identified 54 articles that satisfied the inclusion criteria. These included a single randomized controlled trial, a single systematic review, a single narrative review and a single literature review, with the vast majority having level 2 evidence. Global rating scales, proficiency assessments and written/oral examinations were described as standard current assessment tools. These modalities lack reproducibility and objectivity, neglecting the needs of assessment of endovascular procedures requiring specialized decision making and finger dexterity. Novel methods such as high fidelity simulation and virtual reality promote reproducible and objective assessment methods in the context of endovascular surgery, and have a promising future. Conclusion: While current assessment methods in vascular surgery are widely supported the changing skills required of a vascular surgery trainee warrants a shift in assessment modalities to better align to these requirements. High fidelity simulations show promise, although they require more extensive research to understand their relative merits.

A pilot study of the use of artificial intelligence with high‐fidelity simulations in assessing endovascular procedural competence independent of a human examiner

“…9,10 Nevertheless, a subjective examiner risks bias and diminishes cross-sectional and longitudinal reproducibility. 10 A reproducible, objective and safe assessment method for technical competency in endovascular surgery needs development which may require the support of AI.…”

“…While DOPS assessments have demonstrated reproducibility, reliability and validity in determining differences in skill between trainees in open surgery, 8 poor macroscopic visualization of tissue handling and instrument control in endovascular surgery challenges the ability of the current DOPS assessment in accurately assessing endovascular competence. Several systematic reviews have validated the use of high‐fidelity simulators in training of endovascular surgical skills 9 and have been shown to correlate with global ratings scales 9,10 . Nevertheless, a subjective examiner risks bias and diminishes cross‐sectional and longitudinal reproducibility 10 .…”

“…Several systematic reviews have validated the use of high-fidelity simulators in training of endovascular surgical skills 9 and have been shown to correlate with global ratings scales. 9,10 Nevertheless, a subjective examiner risks bias and diminishes cross-sectional and longitudinal reproducibility. 10 A reproducible, objective and safe assessment method for technical competency in endovascular surgery needs development which may require the support of AI.…”