Surgical Gesture Classification from Video Data

Haro, Benjamín Béjar; Zappella, Luca; Vidal, René

doi:10.1007/978-3-642-33415-3_5

Cited by 39 publications

(3 citation statements)

References 22 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This led to further research in improving temporal learning in MIS with the addition of long short-term memory (LSTM) neural networks, which allows for more efficiency in identifying phases of surgery and tracking surgical instruments. Combining a CNN and LSTM gives the advantage of more efficiently identifying phases of surgery and tracking surgical instruments [30,31]. The EndoNet researchers published follow-up research comparing a CNN with HMM versus a CNN with LTSM [31].…”

Section: Figurementioning

confidence: 99%

Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning

Power,

Burke,

Madden

et al. 2024

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) has the potential to improve healthcare and patient safety and is currently being adopted across various fields of medicine and healthcare. AI and in particular computer vision (CV) are well suited to the analysis of minimally invasive surgical simulation videos for training and performance improvement. CV techniques have rapidly improved in recent years from accurately recognizing objects, instruments, and gestures to phases of surgery and more recently to remembering past surgical steps. Lack of labeled data is a particular problem in surgery considering its complexity, as human annotation and manual assessment are both expensive in time and cost, and in most cases rely on direct intervention of clinical expertise. In this work, a newly collected simulated laparoscopic surgical dataset (LSPD) is presented that will initiate the research in automating this problem and avoiding manual expert assessments. LSPD statistical analyses are given to show similarities and differences between different expertise levels (on Stack, Bands, and Tower Skills). In addition, a 3-dimensional convolutional neural network (3DCNN) is used to classify the experience level of the surgeons, novices, and trainees and is found to achieve good results at distinguishing these, with F1 score of 0.91 and AUC of 0.92.

show abstract

Section: Figurementioning

confidence: 99%

Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning

Power,

Burke,

Madden

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, researchers have used automated performance metrics to predict a patient's postoperative length of stay within a hospital 26 . Another line of research has instead focused on exclusively exploiting live surgical videos from endoscopic cameras to classify surgical activity 4,29 , gestures 5,[30][31][32][33] and skills 6,7,13,34,35 , among other tasks 36,37 . For information on additional studies, we refer readers to a recent review 9 .…”

Section: Previous Workmentioning

confidence: 99%

A vision transformer for decoding surgeon activity from surgical videos

Kiyasseh

Haque

et al. 2023

Nat. Biomed. Eng

View full text Add to dashboard Cite

The intraoperative activity of a surgeon has substantial impact on postoperative outcomes. However, for most surgical procedures, the details of intraoperative surgical actions, which can vary widely, are not well understood. Here we report a machine learning system leveraging a vision transformer and supervised contrastive learning for the decoding of elements of intraoperative surgical activity from videos commonly collected during robotic surgeries. The system accurately identified surgical steps, actions performed by the surgeon, the quality of these actions and the relative contribution of individual video frames to the decoding of the actions. Through extensive testing on data from three different hospitals located in two different continents, we show that the system generalizes across videos, surgeons, hospitals and surgical procedures, and that it can provide information on surgical gestures and skills from unannotated videos. Decoding intraoperative activity via accurate machine learning systems could be used to provide surgeons with feedback on their operating skills, and may allow for the identification of optimal surgical behaviour and for the study of relationships between intraoperative factors and postoperative outcomes.

show abstract

“…At the latter level, which is typically concerned with robotassisted surgery or training and assessment of surgeons, we see research on phase detection (Stauder, 2014) and detailed models of individual tool usage patterns based on sensor data (Ahmadi, 2009). Individual hand motions from video data are automatically identified in (Lin, 2006) and (Haro, 2012). A number of models based on sensor data collected during Cholecystectomies (a highly standardized procedure), were developed in (Blum, 2008), (Bouarfa and Dankelman, 2012), (Bouarfa, 2011), and (Neumuth, 2011).…”

Section: Related Workmentioning

confidence: 99%

Mining Patient Flow Patterns in a Surgical Ward

Back,

Manataki,

Harrison

2020

Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies

View full text Add to dashboard Cite

Surgery is a highly critical and costly procedure, and there is an imperative need to improve the efficiency in surgical wards. Analyzing surgical patient flow and predicting cycle times of different peri-operative phases can help improve the scheduling and management of surgeries. In this paper, we propose a novel approach to mining temporal patterns of surgical patient flow with the use of Bayesian belief networks. We present and compare three classes of probabilistic models and we evaluate them with respect to predicting cycle times of individual phases of patient flow. The results of this study support previous work that surgical times are log-normally distributed. We also show that the inclusion of a clustering pre-processing step improves the performance of our models considerably.

show abstract

Surgical Gesture Classification from Video Data

Cited by 39 publications

References 22 publications

Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning

Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning

A vision transformer for decoding surgeon activity from surgical videos

Mining Patient Flow Patterns in a Surgical Ward

Contact Info

Product

Resources

About