Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery

Kim, Tae Soo; O’Brien, Megan; Zafar, Sidra; Hager, Gregory D.; Sikder, Shameema; Vedula, Satyanarayana S

doi:10.1007/s11548-019-01956-8

Cited by 40 publications

(41 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although studies assessing the performance of real surgery were limited in their number, their proportion has increased since 2018. These studies have investigated procedures across the fields of urology 18 – 20 , general surgery 21 , 22 , otolaryngology 23 , 24 and ophthalmology 25 . Table 2 details the variety of tasks and environments used in the studies included in this review.…”

Section: Resultsmentioning

confidence: 99%

“… • Convolutional Neural Networks. • Recurrent Neural Networks 14 , 16 , 21 , 24 , 25 , 32 , 33 , 63 , 65 , 69 , 78 , 79 , 82 , 83 , 99 , 101 , 103 Convolutional Neural Network (CNN) CNN is an artificial neural network with a “Deep” structure, convolution operation layers and pooling layers. CNN has the ability of representation learning, where it could carry out shift-invariant classification of input information based on its hierarchical structure 105 .…”

Section: Resultsmentioning

confidence: 99%

“… • Multilayer Perceptron. • Recurrent Neural Networks 21 , 24 , 25 , 32 , 33 , 69 , 78 , 79 , 82 , 99 , 101 Recurrent Neural Networks (RNN) The recurrent neural networks are designed for modeling sequential processes. It takes the current observation together with the output of the network in previous state to generates the output.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Machine learning for technical skill assessment in surgery: a systematic review

et al. 2022

View full text Add to dashboard Cite

Accurate and objective performance assessment is essential for both trainees and certified surgeons. However, existing methods can be time consuming, labor intensive, and subject to bias. Machine learning (ML) has the potential to provide rapid, automated, and reproducible feedback without the need for expert reviewers. We aimed to systematically review the literature and determine the ML techniques used for technical surgical skill assessment and identify challenges and barriers in the field. A systematic literature search, in accordance with the PRISMA statement, was performed to identify studies detailing the use of ML for technical skill assessment in surgery. Of the 1896 studies that were retrieved, 66 studies were included. The most common ML methods used were Hidden Markov Models (HMM, 14/66), Support Vector Machines (SVM, 17/66), and Artificial Neural Networks (ANN, 17/66). 40/66 studies used kinematic data, 19/66 used video or image data, and 7/66 used both. Studies assessed the performance of benchtop tasks (48/66), simulator tasks (10/66), and real-life surgery (8/66). Accuracy rates of over 80% were achieved, although tasks and participants varied between studies. Barriers to progress in the field included a focus on basic tasks, lack of standardization between studies, and lack of datasets. ML has the potential to produce accurate and objective surgical skill assessment through the use of methods including HMM, SVM, and ANN. Future ML-based assessment tools should move beyond the assessment of basic tasks and towards real-life surgery and provide interpretable feedback with clinical value for the surgeon.PROSPERO: CRD42020226071

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Machine learning for technical skill assessment in surgery: a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…As suggested by the study conducted by Birkmeyer et al for bariatric surgery [165], surgical skills can be highly correlated to surgical outcome for certain procedures. AI systems have been shown capable of evaluating technical skills using data from either training scenarios [166] or real procedures [167]. Yet, by severely under-utilising the rich information contained in other data sources, the Language of Surgical Dexterity is still not capturing the most complex aspects of surgical decision making.…”

Section: Discussionmentioning

confidence: 99%

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

et al. 2020

View full text Add to dashboard Cite

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.Index Terms-Artificial intelligence, computer assisted interventions, interventional workflow, intra-operative imaging, surgical planning, data fusion, surgical scene understanding, contextaware user interface, machine and deep learning, surgical data science

show abstract

“…In theory, deep learning algorithms can be trained to detect distinct signatures of operative technique that experts intuit, such as speed or confidence of phaco tip movements. 30 Objective skill assessments for specific tasks are already available on cataract simulators with built-in modules, some of which have been shown to discriminate novice from expert surgical proficiency 12 and, once mastered in the simulated setting, may lead to improved live surgical performance by trainees. 31 …”

Section: Discussionmentioning

confidence: 99%

Crowdsourced Assessment of Surgical Skill Proficiency in Cataract Surgery

Paley

Grove

Sekhar

et al. 2021

Journal of Surgical Education

View full text Add to dashboard Cite

OBJECTIVE: To test whether crowdsourced lay raters can accurately assess cataract surgical skills. DESIGN: Two-armed study: independent cross-sectional and longitudinal cohorts. SETTING: Washington University Department of Ophthalmology. PARTICIPANTS AND METHODS: Sixteen cataract surgeons with varying experience levels submitted cataract surgery videos to be graded by 5 experts and 300+ crowdworkers masked to surgeon experience. Cross-sectional study : 50 videos from surgeons ranging from first-year resident to attending physician, pooled by years of training. Longitudinal study : 28 videos obtained at regular intervals as residents progressed through 180 cases. Surgical skill was graded using the modified Objective Structured Assessment of Technical Skill (mOSATS). Main outcome measures were overall technical performance, reliability indices, and correlation between expert and crowd mean scores. RESULTS: Experts demonstrated high interrater reliability and accurately predicted training level, establishing construct validity for the modified OSATS. Crowd scores were correlated with (r = 0.865, p < 0.0001) but consistently higher than expert scores for first, second, and third-year residents (p < 0.0001, paired t-test). Longer surgery duration negatively correlated with training level (r = −0.855, p < 0.0001) and expert score (r = −0.927, p < 0.0001). The longitudinal dataset reproduced cross-sectional study findings for crowd and expert comparisons. A regression equation transforming crowd score plus video length into expert score was derived from the cross-sectional dataset (r 2 = 0.92) and demonstrated excellent predictive modeling when applied to the independent longitudinal dataset (r 2 = 0.80). A group of student raters who had edited the cataract videos also graded them, producing scores that more closely approximated experts than the crowd. CONCLUSIONS: Crowdsourced rankings correlated with expert scores, but were not equivalent; crowd scores overestimated technical competency, especially for novice surgeons. A novel approach of adjusting crowd scores with surgery duration generated a more accurate predictive model for surgical skill. More studies are needed before crowdsourcing can be reliably used for assessing surgical proficiency.

show abstract

Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery

Cited by 40 publications

References 12 publications

Machine learning for technical skill assessment in surgery: a systematic review

Machine learning for technical skill assessment in surgery: a systematic review

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

Crowdsourced Assessment of Surgical Skill Proficiency in Cataract Surgery

Contact Info

Product

Resources

About