Tracking of Retinal Microsurgery Tools Using Late Fusion of Responses from Convolutional Neural Network over Pyramidally Decomposed Frames

Mishra, Krishn Kumar; Sathish, Rachana; Sheet, Debdoot

doi:10.1007/978-3-319-68124-5_31

Cited by 4 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These have also been applied for the analysis of surgical videos, such as instrument presence detection [11, 12], phase recognition [13, 14], tool location [15–17], and tool pose estimation [2, 18]. For example, a cascading model, which consists of a rough location network and a fine‐grained search network, was proposed by Mishra et al [19] to locate the tool tip. In the work of Chen et al [20], a CNN is trained with the datasets labelled by a line segment detector to detect a tool's tip, and then the spatial and temporal context algorithm [21] is utilised to detect the tool in real time.…”

Section: Introductionmentioning

confidence: 99%

Real‐time surgical instrument detection in robot‐assisted surgery using a convolutional neural network cascade

Zhao¹,

Cai²,

Chang³

et al. 2019

Healthcare Technology Letters

View full text Add to dashboard Cite

Surgical instrument detection in robot-assisted surgery videos is an import vision component for these systems. Most of the current deep learning methods focus on single-tool detection and suffer from low detection speed. To address this, the authors propose a novel frameby-frame detection method using a cascading convolutional neural network (CNN) which consists of two different CNNs for real-time multi-tool detection. An hourglass network and a modified visual geometry group (VGG) network are applied to jointly predict the localisation. The former CNN outputs detection heatmaps representing the location of tool tip areas, and the latter performs bounding-box regression for tool tip areas on these heatmaps stacked with input RGB image frames. The authors' method is tested on the publicly available EndoVis Challenge dataset and the ATLAS Dione dataset. The experimental results show that their method achieves better performance than mainstream detection methods in terms of detection accuracy and speed.

show abstract

Section: Introductionmentioning

confidence: 99%

Real‐time surgical instrument detection in robot‐assisted surgery using a convolutional neural network cascade

Zhao¹,

Cai²,

Chang³

et al. 2019

Healthcare Technology Letters

View full text Add to dashboard Cite

show abstract

“…To evaluate the performance of the method, we compared it to three other methods. The methods we chose are: locating instrument tip directly by Convnet2-based traditional sliding window method (SWM) [24], tracking with an active filter (AF) [25], and scale invariant deep learning-based detection approach (SIDL) [16]. These methods were all implemented in Matlab.…”

Section: Data Preparation and Cnn Trainingmentioning

confidence: 99%

“…Moreover, to get a more precise location, more than one convolutional neural network may be used in some detection methods. Mishra et al [16] proposed a deep learning approach based on late fusion CNN responses over pyramidally decomposed frames to locate the instrument tip. The network used above is deep and it is hard to train.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convolutional neural network-based surgical instrument detection

Cai¹,

Zhao²

2020

THC

View full text Add to dashboard Cite

BACKGROUND: Minimally invasive surgery (MIS), unlike open surgery in which surgeons can perform surgery directly, is performed using miniaturized instruments with indirect but careful observation of the surgical site. OBJECTIVE: Instrument detection is a crucial requirement in conventional and robot-assisted MIS, which can also be very useful during surgical training. In this paper, we propose a novel framework of using two three-layer convolutional neural networks (CNNs) in a series to detect surgical instrument in in-vivo video frames. METHODS: The two convolutional neural networks proposed in this paper have different tasks. (i) The former CNN is trained to detect the edges points of the instrument shaft directly from images patches. (ii) The latter is trained to locate the instrument tip also from images patches after the former detection finishes. RESULTS: We validated our method on the publicly available EndoVisSub dataset and a standard dataset, and it detected tools with an accuracy of 91.2% and 75% respectively. CONCLUSION: Our two-step detection method achieves better performance than other existing approaches in terms of detection accuracy.

show abstract

An Extremely Fast and Precise Convolutional Neural Network for Recognition and Localization of Cataract Surgical Tools

Zang

Bian

Wang

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Tracking of Retinal Microsurgery Tools Using Late Fusion of Responses from Convolutional Neural Network over Pyramidally Decomposed Frames

Cited by 4 publications

References 10 publications

Real‐time surgical instrument detection in robot‐assisted surgery using a convolutional neural network cascade

Real‐time surgical instrument detection in robot‐assisted surgery using a convolutional neural network cascade

Convolutional neural network-based surgical instrument detection

An Extremely Fast and Precise Convolutional Neural Network for Recognition and Localization of Cataract Surgical Tools

Contact Info

Product

Resources

About