Decoupling channel count from field of view and spatial resolution in single-sensor imaging systems for fluorescence image-guided surgery

Abstract

. Significance Near-infrared fluorescence image-guided surgery is often thought of as a spectral imaging problem where the channel count is the critical parameter, but it should also be thought of as a multiscale imaging problem where the field of view and spatial resolution are similarly important. Aim Conventional imaging systems based on division-of-focal-plane architectures suffer from a strict relationship between the channel count on one hand and the field of view and spatial resolution on the other, but bioinspired imaging systems that combine stacked photodiode image sensors and long-pass/short-pass filter arrays offer a weaker tradeoff. Approach In this paper, we explore how the relevant changes to the image sensor and associated image processing routines affect image fidelity during image-guided surgeries for tumor removal in an animal model of breast cancer and nodal mapping in women with breast cancer. Results We demonstrate that a transition from a conventional imaging system to a bioinspired one, along with optimization of the image processing routines, yields improvements in multiple measures of spectral and textural rendition relevant to surgical decision-making. Conclusions These results call for a critical examination of the devices and algorithms that underpin image-guided surgery to ensure that surgeons receive high-quality guidance and patients receive high-quality outcomes as these technologies enter clinical practice.

Cell-Membrane Coated Nanoparticles for Tumor Delineation and Qualitative Estimation of Cancer Biomarkers at Single Wavelength Excitation in Murine and Phantom Models

“…A solution is posed by an 18-band snapshot hyperspectral imaging system that was previously evaluated for sentinel lymph node dissection with multiple near-infrared fluorophores but is equally applicable for tumor detection with a collection of cancer biomarkers . By permitting multiple wavelengths to be probed at each pixel, this snapshot hyperspectral imaging system preserves the high frame rates that are possible with color imaging systems while facilitating a balance between field of view and spatial resolution that has been demonstrated with similarly constructed imaging systems. , These features enable the imaging system to capture more information at a faster rate than laboratory instruments without sacrificing the benefit of image fidelity that is offered by those instruments. Spectral unmixing of two RBC-SPNs was conducted using an 18-band snapshot hyperspectral imaging system designed for near-infrared fluorescence image-guided surgery (Figure S35).…”

Convolutional neural network advances in demosaicing for fluorescent cancer imaging with color–near-infrared sensors

“…However, similar to the Bayer filter in color imaging, 21 pixelated NIR–color cameras experience a reduction in spatial resolution. 31 , 32 Fortunately, this challenge is not new to color imaging, and various demosaicing techniques have been developed to mitigate spatial resolution loss. 33 Color demosaicing techniques, essential for converting raw sensor data into full-color images, can be broadly classified into three main approaches: interpolation, dictionary-based, and learning-based.…”

“…However, similar to the Bayer filter in color imaging, 21 pixelated NIR–color cameras experience a reduction in spatial resolution 31 , 32 . Fortunately, this challenge is not new to color imaging, and various demosaicing techniques have been developed to mitigate spatial resolution loss 33 .…”