Photometric stereo endoscopy

Abstract

Abstract. While color video endoscopy has enabled wide-field examination of the gastrointestinal tract, it often misses or incorrectly classifies lesions. Many of these missed lesions exhibit characteristic three-dimensional surface topographies. An endoscopic system that adds topographical measurements to conventional color imagery could therefore increase lesion detection and improve classification accuracy. We introduce photometric stereo endoscopy (PSE), a technique which allows high spatial frequency components of surface topography to be acquired simultaneously with conventional two-dimensional color imagery. We implement this technique in an endoscopic form factor and demonstrate that it can acquire the topography of small features with complex geometries and heterogeneous optical properties. PSE imaging of ex vivo human gastrointestinal tissue shows that surface topography measurements enable differentiation of abnormal shapes from surrounding normal tissue. Together, these results confirm that the topographical measurements can be obtained with relatively simple hardware in an endoscopic form factor, and suggest the potential of PSE to improve lesion detection and classification in gastrointestinal imaging. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

State‐of‐the‐art in Multi‐Light Image Collections for Surface Visualization and Analysis

“…The application of multi‐light visualization in the area of medical science strives to increase readability of medical images. It is mostly used in the type of examination where the internal organ such as the esophagus, stomach, and upper part of the small intestine is visually inspected, often in real‐time during examinations [PLG∗13, CB12]. This is typically achieved by modifying existing instruments (e.g., endoscopes [DGL∗14]) to perform multi‐light acquisitions, and by fusing several images to help in visualization.…”

“…This is typically achieved by modifying existing instruments (e.g., endoscopes [DGL∗14]) to perform multi‐light acquisitions, and by fusing several images to help in visualization. The typical approach is to reconstruct normals and remove highlights in the visualization [PLG∗13]. By using a simple MLIC capture with few light sources and single image enhanced visualization it is possible to assist doctors in the understanding of the 3D structure of anatomical parts in endoscopic views [RTF∗04].…”

The role of technology in minimally invasive surgery: state of the art, recent developments and future directions

“…The 3D data can be recovered from non-stereo cameras through image processing algorithms such as computational or photometric stereo, which extract depth information and allow a 3D image to be reconstructed and displayed 17 18. It is important to note that, while 3D visualisation improves performance and decreases operation time, high-definition (HD) images are still necessary for optimal clinical results 19.…”

Content-based processing and analysis of endoscopic images and videos: A survey

“…However, again some basic assumptions of generic SfS do not hold for endoscopic images, hence adaptations are necessary to obtain acceptable results [43,171,269]. The generalization of SfS to multiple light sources is referred to as photometric stereo and has also been proposed for 3D reconstruction of endoscopic videos [42,178]. This techniques requires hardware modifications to obtain different illumination conditions.…”