Robotic Objective Structured Assessment of Technical Skills

Tarr, Megan E.; Rivard, Colleen; Petzel, A; Summers, Sondra; Rickey, Leslie; Denman, Mary Anna; Harders, Regina; Durazo‐Arvizu, Ramón; Kenton, Kimberly

doi:10.1097/spv.0000000000000067

Cited by 13 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A total of 10 out of 13 (79.6%) global tools reported relationships to other variables by either comparing novices to experts (construct validity) or showing significant correlation between scoring outcomes and other performance assessment tools, considered the gold standard (concurrent validity) 22–24,27,28,30,35,36,38–40,57,58,61–77 . Nine out of 13 (69.2%) procedure‐specific tool studies 31–33,42–44,46–50,60 showed construct or concurrent validity.…”

Section: Resultsmentioning

confidence: 99%

“…A total of 10 out of 13 (79.6%) global tools reported relationships to other variables by either comparing novices to experts (construct validity) or showing significant correlation between scoring outcomes and other performance assessment tools, considered the gold standard (concurrent validity). 22 , 23 , 24 , 27 , 28 , 30 , 35 , 36 , 38 , 39 , 40 , 57 , 58 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 Nine out of 13 (69.2%) procedure‐specific tool studies 31 , 32 , 33 , 42 , 43 , 44 , 46 , 47 , 48 , 49 , 50 , 60 showed construct or concurrent validity. Nine out of 11 (81.8%) APM showed construct or criterion validity.…”

Section: Resultsmentioning

confidence: 99%

“…outcomes and other performance assessment tools, considered the gold standard (concurrent validity). [22][23][24]27,28,30,35,36,[38][39][40]57,58,[61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76][77] Nine out of 13 (69.2%) procedure-specific tool studies [31][32][33][42][43][44][46][47][48][49][50]60 showed construct or concurrent validity.…”

Section: Ta B L Ementioning

confidence: 99%

See 2 more Smart Citations

Objective assessment tools in laparoscopic or robotic‐assisted gynecological surgery: A systematic review

Tesfai,

Nagi,

Morrison

et al. 2024

Acta Obstet Gynecol Scand

View full text Add to dashboard Cite

IntroductionThere is a growing emphasis on proficiency‐based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.Material and methodsA systematic search (1989–2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single‐group (with pre‐ and post‐training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta‐analysis.ResultsA total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure‐specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.ConclusionsExisting objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

show abstract

Section: Resultsmentioning

confidence: 99%

“…A total of 10 out of 13 (79.6%) global tools reported relationships to other variables by either comparing novices to experts (construct validity) or showing significant correlation between scoring outcomes and other performance assessment tools, considered the gold standard (concurrent validity). 22 , 23 , 24 , 27 , 28 , 30 , 35 , 36 , 38 , 39 , 40 , 57 , 58 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 Nine out of 13 (69.2%) procedure‐specific tool studies 31 , 32 , 33 , 42 , 43 , 44 , 46 , 47 , 48 , 49 , 50 , 60 showed construct or concurrent validity. Nine out of 11 (81.8%) APM showed construct or criterion validity.…”

Section: Resultsmentioning

confidence: 99%

Section: Ta B L Ementioning

confidence: 99%

See 1 more Smart Citation

Objective assessment tools in laparoscopic or robotic‐assisted gynecological surgery: A systematic review

Tesfai,

Nagi,

Morrison

et al. 2024

Acta Obstet Gynecol Scand

View full text Add to dashboard Cite

show abstract

Incorporating resident/fellow training into a robotic surgery program

Vetter

Green

Martino

et al. 2015

Journal of Surgical Oncology

View full text Add to dashboard Cite

With the rapid uptake of the robotic approach in gynecologic surgery, a thorough understanding of the technology, including its uses and limitations, is critical to maximize patient outcomes and safety. This review discusses the role of training modalities and development of curricula for robotic surgery. Furthermore, methods for incorporating the entire surgical team and the process of credentialing/maintaining privileges are described.

show abstract

Analysis of executional and procedural errors in dry‐lab robotic surgery experiments

Cantrell

Schenkman

et al. 2022

Robotics Computer Surgery

View full text Add to dashboard Cite

Background Analysing kinematic and video data can help identify potentially erroneous motions that lead to sub‐optimal surgeon performance and safety‐critical events in robot‐assisted surgery. Methods We develop a rubric for identifying task and gesture‐specific executional and procedural errors and evaluate dry‐lab demonstrations of suturing and needle passing tasks from the JIGSAWS dataset. We characterise erroneous parts of demonstrations by labelling video data, and use distribution similarity analysis and trajectory averaging on kinematic data to identify parameters that distinguish erroneous gestures. Results Executional error frequency varies by task and gesture, and correlates with skill level. Some predominant error modes in each gesture are distinguishable by analysing error‐specific kinematic parameters. Procedural errors could lead to lower performance scores and increased demonstration times but also depend on surgical style. Conclusions This study provides insights into context‐dependent errors that can be used to design automated error detection mechanisms and improve training and skill assessment.

show abstract

Robotic Objective Structured Assessment of Technical Skills

Abstract: Overall, the residents' robotic skills improved after participating in a dry laboratory curriculum; however, robotic availability, duty hour restrictions, and clinical responsibilities limit the curriculum implementation.

Cited by 13 publications

References 12 publications

Objective assessment tools in laparoscopic or robotic‐assisted gynecological surgery: A systematic review

Objective assessment tools in laparoscopic or robotic‐assisted gynecological surgery: A systematic review

Incorporating resident/fellow training into a robotic surgery program

Analysis of executional and procedural errors in dry‐lab robotic surgery experiments

Contact Info

Product

Resources

About