An endoscopic structured light system using multispectral detection

Abstract

Purpose In clinical examinations, the tissue surface topology is an important feature for detecting the tissue pathology and implementing augmented reality. We have previously presented a miniaturised structured light (SL) system for recovery of tissue surface shape in minimally invasive surgery (MIS), based on a flexible multispectral structured illumination probe (1.9 mm diameter) [1]. This paper reports further hardware and analytical developments to improve the light pattern decoding result and increase the reconstruction accuracy.Methods The feasibility of using an 8-band multispectral camera with higher pattern-colour discrimination ability than normal RGB camera in this system was studied. Additionally, the "normalized cut" algorithm was investigated to improve pattern segmentation.Results The whole SL system was evaluated by phantom and in vivo experiments. Higher pattern identification performance than that of an RGB camera was recorded by using the multispectral camera (average precision > 97%, average sensitivity > 62%). An average of 0.88±0.67 mm reconstruction error was achieved using the proposed pattern decoding method on a heart phantom at a working distance of approximately 10 cm. ConclusionsThe experiment showed the superiority of the multispectral camera over the RGB camera in the spot identification step. The proposed pattern decoding algorithm underwent evaluations using different experiments, showing that it provided promising reconstruction results.The potential of using this system in MIS environments has been demonstrated.

Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging

“…Previously, a SL-enabled 3D tissue surface shape and hyperspectral imaging system was presented [4,5]. This used an optical fiber bundle with the fibers arranged in linear and circular arrays, respectively, at either end ( Fig.…”

Probe-Based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks

“…Table 1 shows that the pattern decoding algorithm functions robustly with both the phantom and even some challenging ex vivo data, with feature matching sensitivity higher than 0.8, and precision 0.99. Compared with previous work [13], this shows a much higher sensitivity for feature matching due to the high spot detection accuracy achievable with FCN. Meanwhile, the near real-time performance of FCN guarantees its real-world practicality.…”

“…Previous work has shown that the projected spot patterns are robust to changes in background albedo, due to their narrow bandwidth [11]. It was also found that, given an accurate calibration and perfect feature matching, reconstruction error can reach 0.7 mm at a working distance of ~10 cm [13]. Thus, the reconstruction accuracy mainly depends on the pattern decoding results.…”

Endoscopic vitreoretinal surgery: principles, applications and new directions

“…Image size, contrast and color balance adjustment have been available [5]. Additional computer techniques for real-time enhancement of surgical images include spatial frequency modulation (filtering) [72, 73], tissue topology and texture recognition [74], adaptive illumination adjustment [75] and image stabilization [76].…”