Crowdsourcing Assessment of Surgeon Dissection of Renal Artery and Vein During Robotic Partial Nephrectomy: A Novel Approach for Quantitative Assessment of Surgical Performance

Abstract

We conclude that crowdsourced assessment of qualitative performance ratings may be an alternative and/or adjunct to surgical experts' ratings and would provide a rapid scalable solution to triage technical skills.

Implementing assessments of robot‐assisted technical skill in urological education: a systematic review and synthesis of the validity evidence

“…The Global Evaluative Assessment of Robotic Skills (GEARS) tool, developed by Goh et al. , has been applied to urological assessments on multiple occasions , and has the strongest validity argument supporting its use in the assessment of robotic skill. Its generic framework has allowed it to become a widely accepted method of assessment across multiple procedures and even across specialties .…”

“…The vast majority of literature using the GEARS score has found it to be a reliable assessment method . However, a study of robotic renal hilar dissection using oriented expert raters showing poor internal consistency , and Hung et al. found that while trainee self‐assessments and faculty evaluations correlated weakly, inter‐faculty reliability was better when assessing residents [intraclass correlation (ICC) = 0.77] and fellows (ICC = 0.45).…”

“…Table 1 summarises the validity evidence supporting the seven non-time-based technical skill assessment tools used in urological RAS. The Global Evaluative Assessment of Robotic Skills (GEARS) tool, developed by Goh et al [4], has been applied to urological assessments on multiple occasions [4,[11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29], and has the strongest validity argument supporting its use in the assessment of robotic skill. Its generic framework has allowed it to become a widely accepted method of assessment across multiple procedures and even across specialties [4,15,19].…”

Objective Assessment of Surgical Technical Skill and Competency in the Operating Room

“…Although GRS accurately distinguish surgeons with different levels of experience (22), integrating them into surgical training curricula has been limited, in part due to resource constraints because they require an expert observer (21, 25). Recent innovations using crowdsourcing may be an efficient alternative to observation by experts, but their role in surgical training curricula is still uncertain (26–30). Other, less resource intensive approaches, such as in-training evaluations and examinations, or surrogates, such as surgical volume (31–33), do not correlate with objective assessments of surgical technical skill (34).…”

Educational Crowdsourcing: Developing RadExam

“…This could not have been accomplished without crowdsourcing from the academic radiology community. Examples of crowdsourcing can be found across a wide variety of disciplines, including medical education (eg, Radiopaedia) and for the evaluation of trainees [6][7][8][9].…”