Data-Derived Models for Segmentation with Application to Surgical Assessment and Training

Abstract

Abstract. This paper addresses automatic skill assessment in robotic minimally invasive surgery. Hidden Markov models (HMMs) are developed for individual surgical gestures (or surgemes) that comprise a typical bench-top surgical training task. It is known that such HMMs can be used to recognize and segment surgemes in previously unseen trials [1]. Here, the topology of each surgeme HMM is designed in a data-driven manner, mixing trials from multiple surgeons with varying skill levels, resulting in HMM states that model skill-specific sub-gestures. The sequence of HMM states visited while performing a surgeme are therefore indicative of the surgeon's skill level. This expectation is confirmed by the average edit distance between the state-level "transcripts" of the same surgeme performed by two surgeons with different expertise levels. Some surgemes are further shown to be more indicative of skill than others.

Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery

“…In laparoscopic surgeries, skill level is evaluated by measuring force and motion data [3]. The promising results with tele-operated robotic systems [2][6] [7] show that Hidden Markov Models (HMM) enable us to recognize skill level and subtasks from motion data. Results in [8] show that rotated view of camera in laparoscopic surgeries increases the complexity of the task.…”

Modeling and Segmentation of Surgical Workflow from Laparoscopic Video

“…One way to handle this would be to use more complex HMM topologies that take into account more of the semantics of the data. This could be achieved using HMMs that derive their topology from data as done by [14]. One advantage of DTW is that a warping path is obtained that assigns every time step of a surgery to a time step of the average model.…”

Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation

“…These methods are generally easy to implement, but lack a detailed description of the surgical procedure. Another approach is to use statistical models to decompose a surgical task into a series of pre-defined surgical gestures or surgemes [6][7][8][9][10][11]. For example, in a suturing task, the surgemes can be 'insert a needle', 'grab a needle', 'position a needle', etc.…”

“…However, parameter learning may not be robust when the data is high-dimensional because of the large number of parameters to be estimated. To address this issue, [9] combines Gaussian HMMs (G-HMMs) with Linear Discriminant Analysis (LDA) [12], while [10] proposes several variations of HMMs, such as Factor Analyzed HMMs (FA-HMMs), and Switched Linear Dynamical Systems (SLDSs), which model the observations as being generated from a lower-dimensional latent space. However, the observation model is still Gaussian, which may not be rich enough to capture the variability of complex gestures.…”