Deep Smoke Removal from Minimally Invasive Surgery Videos

Abstract

During video-guided minimally invasive surgery, quality of frames may be degraded severely by cauterization-induced smoke and condensation of vapor. This degradation of quality creates discomfort for the operating surgeon, and causes serious problems for automatic follow-up processes such as registration, segmentation and tracking. This paper proposes a novel deep neural network based smoke removal solution that is able to enhance the quality of surgery video frames in real-time. It employs synthetically generated training dataset including smoke embedded and clean reference versions. Results calculated on the test set indicate that our network outperforms previous defogging methods in terms of quantitative and qualitative measures. While eliminating apparent smoke, it also successfully preserves the natural appearance of tissue surface. To the best of our knowledge, the presented method is the first deep neural network based approach for the surgical field smoke removal problem.

Desmoking Laparoscopy Surgery Images Using an Image-to-Image Translation Guided by an Embedded Dark Channel

“…To analyze the compared methods' performance under different smoke intensities, we define three smoke levels as a function of the parameter smoke intensity in (8) [15]; it uses a convolutional architecture with multi-scale kernels. The second is [19], where the authors use an unsupervised learning approach based on U-Net structure.…”

“…(a) Traditional image processing techniques [7]- [11], (b) Physics-model-based methods, especially the atmospheric scattering model and the dark channel prior (DCP) [2], [12], [13], (c) Artificial intelligence (AI) methods based on convolu- tional neural networks and generative networks [15]- [19], [22].…”

“…The state-of-the-art in artificial-intelligence-based solutions is as follows. In [15] a deep neural network is implemented in real-time to enhance the quality of surgery video frames trying to eliminate the smoke presented in the image. The authors of [16] propose an unsupervised imageto-image translation with a generative adversarial network (GAN) architecture; they use a perceptual image quality metric for the loss function.…”

Multiscale deep desmoking for laparoscopic surgery

“…Image processing based smoke removal is a recent topic and there are few works proposed. [5][6][7][8][9][10][11] In these papers, the approaches can be classified into traditional approaches [5][6][7][8][9] and deep learning approaches 10,11 which will be discussed in this section.…”

“…Deep learning approaches: The first deep learning desmoking approach is proposed in, 10 Bolkar et al propose to generate a synthetic dataset by Perlin noise, 17 then the dataset is used to fine tuning a dehazing network AOD-Net. 18 The computational speed of this method reaches 20 fps for size of 512× 512 color videos.…”