Automated surgical skill assessment in RMIS training

Zia, Aneeq; Essa, Irfan

doi:10.1007/s11548-018-1735-5

Cited by 108 publications

(109 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this approach does leverage the gesture boundaries for training purposes, our method is much more accurate without the need to manually segment each surgical trial into finer gestures. [24] introduced approximate Entropy (ApEn) to generate characteristics from each surgical task, which are later given to a classical nearest neighbor classifier with a cosine similarity metric. Although ApEn and FCN achieved state-of-the-art results with 100% accuracy for the first two surgical tasks, it is still not obvious how ApEn could be used to give feedback for the trainee after finishing his/her training session.…”

Section: Resultsmentioning

confidence: 99%

“…By localizing, for example, discriminative behaviors specific to a skill level, observers can start to understand motion patterns specific to certain class of surgeons. To further improve themselves (the novice [19] 97.4 n/a n/a 96.2 n/a n/a 94.4 n/a n/a ApEn [24] 100 n/a 0.59 100 n/a 0.45 99.9 n/a 0.66 Sax-Vsm [4] 89.7 86.7 n/a 96.3 95.8 n/a 61.1 53.3 n/a CNN [21] 93.4 n/a n/a 89.9 n/a n/a 84.9 n/a n/a FCN (proposed) 100 100 0.60 100 100 0.57 92.1 93.2 0.65 By generating a heatmap from the CAM, we can see in Fig. 2 how it is indeed possible to visualize the feedback for the trainee.…”

Section: Resultsmentioning

confidence: 99%

“…The second type of problem in surgical skill evaluation is to train a model that predicts the modified OSATS score for a certain surgical trial. For example, [24] extended their ApEn model to predict the OSATS score, also known as global rating score (GRS). Interestingly, the latter extension to a regression model instead of a classification one enabled the authors to propose a technique that provides interpretability of the model's decision, whereas our neural network provides an explanation for both classification and regression tasks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

Purpose Manual feedback from senior surgeons observing less experienced trainees is a laborious task that is very expensive, time-consuming and prone to subjectivity. With the number of surgical procedures increasing annually, there is an unprecedented need to provide an accurate, objective and automatic evaluation of trainees' surgical skills in order to improve surgical practice. Methods In this paper, we designed a convolutional neural network (CNN) to classify surgical skills by extracting latent patterns in the trainees' motions performed during robotic surgery. The method is validated on the JIGSAWS dataset for two surgical skills evaluation tasks: classification and regression. Results Our results show that deep neural networks constitute robust machine learning models that are able to reach new competitive state-of-the-art performance on the JIGSAWS dataset. While we leveraged from CNNs' efficiency, we were able to minimize its blackbox effect using the class activation map technique. Conclusions This characteristic allowed our method to automatically pinpoint which parts of the surgery influenced the skill evaluation the most, thus allowing us to explain a surgical skill classification and provide surgeons with a novel personalized feedback technique. We believe this type of interpretable machine learning model could integrate within "Operation Room 2.0"

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Using these features, they train various machine learning classifiers to distinguish between expert and novice. Zia and Essa [33] evaluate texture features, frequency-based features, and entropy-based features extracted from robot kinematic data. For classifying surgeons into expert, intermediate, or novice, they employ a nearest neighbor classifier after dimensionality reduction using principal component analysis.…”

mentioning

confidence: 99%

Video-based surgical skill assessment using 3D convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

Purpose: A profound education of novice surgeons is crucial to ensure that surgical interventions are effective and safe. One important aspect is the teaching of technical skills for minimally invasive or robot-assisted procedures. This includes the objective and preferably automatic assessment of surgical skill.Recent studies presented good results for automatic, objective skill evaluation by collecting and analyzing motion data such as trajectories of surgical instruments. However, obtaining the motion data generally requires additional equipment for instrument tracking or the availability of a robotic surgery system to capture kinematic data. In contrast, we investigate a method for automatic, objective skill assessment that requires video data only. This has the advantage that video can be collected effortlessly during minimally invasive and robot-assisted training scenarios. Methods: Our method builds on recent advances in deep learning-based video classification. Specifically, we propose to use an inflated 3D ConvNet to classify snippets, i.e., stacks of a few consecutive frames, extracted from surgical video. The network is extended into a Temporal Segment Network during training. Results: We evaluate the method on the publicly available JIGSAWS dataset, which consists of recordings of basic robot-assisted surgery tasks performed on a dry lab bench-top model. Our approach achieves high skill classification accuracies ranging from 95.1% to 100.0%. Conclusions: Our results demonstrate the feasibility of deep learning-based assessment of technical skill from surgical video. Notably, the 3D ConvNet is able to learn meaningful patterns directly from the data, alleviating the need for man-

show abstract

“…So far, few studies have proposed methods for specifying the surgemes with relatively poor performance in a task. Recently, Zia et al have proposed to use four different holistic features derived from robot kinematic data in overall skill level classification. Moreover, their work has mentioned the use of discrete cosine transform (DCT) to generate “task highlights.” Task highlights are the surgemes that have the most impact on the overall skill evaluation score.…”

Section: Introductionmentioning

confidence: 99%

An automatic skill evaluation framework for robotic surgery training

Peng

Yuan

Liu

et al. 2018

Robotics Computer Surgery

View full text Add to dashboard Cite

Background To provide feedback to surgeons in robotic surgery training, many surgical skill evaluation methods have been developed. However, they hardly focus on the performance of the surgical motion segments. This paper proposes a method of specifying a trainee's skill weakness in the surgical training. Methods This paper proposed an automatic skill evaluation framework by comparing the trainees' operations with the template operation in each surgical motion segment, which is mainly based on dynamic time warping (DTW) and continuous hidden Markov model (CHMM). Results The feasibility of this proposed framework has been preliminarily verified. For specifying the skill weakness in instrument handling and efficiency, the result of this proposed framework was significantly correlated with that of manual scoring. Conclusion The automatic skill evaluation framework has shown its superiority in efficiency, objectivity, and being targeted, which can be used in robotic surgery training.

show abstract

Automated surgical skill assessment in RMIS training

Cited by 108 publications

References 20 publications

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Video-based surgical skill assessment using 3D convolutional neural networks

An automatic skill evaluation framework for robotic surgery training

Contact Info

Product

Resources

About