Crowdsourcing: A Valid Alternative to Expert Evaluation of Robotic Surgery Skills

Polin,; Siddiqui, NY; Brown, C. T. A.; Hesham, Helai; Comstock, BA; Lendvay, T.S.; Martino, M.A.

doi:10.1016/j.jmig.2015.08.062

Cited by 10 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application of peer assessment of clinical case studies (Heylings and Stefani, 1997) and work habits and interprofessional skills (Spandorfer et al, 2014), and well as innovative approaches focused on crowd‐sourcing for assessment and feedback (Lendvay et al, 2015; Polin et al, 2016) suggests acceptance of non‐expert assessors in the anatomical sciences and does have some overlap with some of the ideas inherent in assessment literacy.…”

Section: Discussionmentioning

confidence: 99%

Rethinking Assessment Design: Evidence‐Informed Strategies to Boost Educational Impact in the Anatomical Sciences

Ryan

Wilkinson

2021

Anatomical Sciences Ed

View full text Add to dashboard Cite

University assessment is in the midst of transformation. Assessments are no longer designed solely to determine that students can remember and regurgitate lecture content, nor in order to rank students to aid with some future selection process. Instead, assessments are expected to drive, support, and enhance learning and to contribute to student self‐assessment and development of skills and attributes for a lifetime of learning. While traditional purposes of certifying achievement and determining readiness to progress remain important, these new expectations for assessment can create tensions in assessment design, selection, and deployment. With the recognition of these tensions, three contemporary approaches to assessment in medical education are described. These approaches include careful consideration of the educational impact of assessment—before, during (test or recall enhanced learning) and after assessments; development of student (and staff) assessment literacy; and planning of cohesive systems of assessment (with a range of assessment tools) designed to assess the various competencies demanded of future graduates. These approaches purposefully straddle the cross purposes of assessment in modern health professions education. The implications of these models are explored within the context of medical education and then linked with contemporary work in the anatomical sciences in order to highlight current synergies and potential future innovations when using evidence‐informed strategies to boost the educational impact of assessments.

show abstract

Section: Discussionmentioning

confidence: 99%

Rethinking Assessment Design: Evidence‐Informed Strategies to Boost Educational Impact in the Anatomical Sciences

Ryan

Wilkinson

2021

Anatomical Sciences Ed

View full text Add to dashboard Cite

show abstract

“…GEARS contains performance anchors and five rating scales (scored 1–5), with a maximum total score of 25. The GEARS scores of the trainees’ robotic video recordings were an average of three scores assessed by external reviewers via an anonymous crowd‐sourced platform (C‐SATS) 34 …”

Section: Methodsmentioning

confidence: 99%

“…C‐SATS provides expert‐based, crowd‐sourced, anonymized, surgical video review that offers GEARS assessment. This validated 34 service has demonstrated the ability to improve surgeon technical skills and ultimately improve patient outcomes. Although C‐SATS is available for TORS and H&N surgeons, the resulting assessment is based on GEARS, which has unproven applicability to TORS.…”

Section: Introductionmentioning

confidence: 99%

Transoral Robotic Surgical Proficiency Via Real‐Time Tactile Collision Awareness System

et al. 2020

View full text Add to dashboard Cite

In 2009, the Food and Drug Administration approved the use of the surgical robotic system for removal of benign and malignant conditions of the upper aerodigestive tract. This novel application of robotic-assisted surgery, termed transoral robotic surgery (TORS), places robotic instruments and camera system through the mouth to reach recessed areas of the pharynx and larynx. Over the successive decade, there was a rapid adoption of TORS with a surgical growth rate that continues to increase. Despite the rapid clinical acceptance, the field of TORS has not yet seen substantive changes or advances in the technical shortcomings, the lack of which has restricted objective TORS-specific surgical skills assessment as well as subsequent skills improvement efforts. One of the primary technical challenges of TORS is operating in a confined space, where the robotic system is maneuvered within the restrictive boundaries of the mouth and throat. Due to these confined boundaries of the pharynx, instruments can frequently collide with anatomic structures such as teeth and bone, producing anatomic collisions. Therefore, we hypothesized that anatomic collisions negatively impact TORS surgical performance. Secondarily, we hypothesized that avoidance of unwanted anatomic collisions could improve TORS surgical proficiency. Methods: Design and fidelity testing for a custom TORS training platform with an integrated anatomic collision-sensing system providing real-time tactile feedback is described. Following successful platform assembly and testing, validation study using the platform was carried through prospective surgical training with trial randomization. Twenty otolaryngology-head and neck surgery residents, each trainee performing three discrete mock surgical trials (n = 60), performed the initial system validation. Ten of the 20 residents were randomized to perform the surgical trials utilizing the real-time feedback system. The remaining 10 residents were randomized to perform the surgical trials without the feedback system, although the system still could record collision data. Surgical proficiency was measured by Global Evaluative Assessment of Robotic Skills (GEARS) score, time to completion, and tumor resection scores (categorical scale ranging 0-3, describing the adequacy of resection). Results: Major anatomic collisions (greater than 5N of force) negatively affected GEARS robotic skills. A mixed model analysis demonstrated that for every additional occurrence of a major collision, GEARS robotic skills assessment score would decrease by 0.29 points (P = .04). Real-time collision awareness created significantly fewer major (> 5 N) anatomic collisions with the tactile feedback system active (n = 30, mean collisions = 2.9 ± 4.2) as compared with trials without tactile feedback (n = 30, mean collisions = 12.53 ± 23.23) (P < .001). The second assessment measure of time to completion was unaffected by the presence of collisions or by the use of tactile feedback system. The third proficiency assessment was measured with tumor r...

show abstract

“…Scoring scales have been matched with quality registry, and surgical skills measured by these scales have shown significant association with the rate of complications 19 . Application of the scoring scales for assessment of surgical skills has been shown to be reliable in the hands of experts (surgeons), nonexperts (crowdsourcing), 20 and by automated software programs 21 . There is an emerging trend for crowdsourcing as a quick and cheap method for assessment of technical skills.…”

Section: Definitionsmentioning

confidence: 99%

Educational value of surgical telementoring

Butt

Augestad

2021

Journal of Surgical Oncology

View full text Add to dashboard Cite

Educating surgeons is a time‐consuming process. In addition to theoretical knowledge, the practical tasks of surgical procedures must be mastered. Translation of such knowledge from mentor to mentee may be efficiently done by surgical telementoring (ST). This is a review on surgical telementoring. Recent technological advances have made this tool in surgical education more available and applicable but future applications of ST have to be wisely guided by high‐quality trials.

show abstract

Crowdsourcing: A Valid Alternative to Expert Evaluation of Robotic Surgery Skills

Cited by 10 publications

References 0 publications

Rethinking Assessment Design: Evidence‐Informed Strategies to Boost Educational Impact in the Anatomical Sciences

Rethinking Assessment Design: Evidence‐Informed Strategies to Boost Educational Impact in the Anatomical Sciences

Transoral Robotic Surgical Proficiency Via Real‐Time Tactile Collision Awareness System

Educational value of surgical telementoring

Contact Info

Product

Resources

About