Light source calibration for multispectral imaging in surgery

Abstract

Purpose Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions. Methods The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component. Results Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision. Conclusion Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery.

Methods and open-source toolkit for analyzing and visualizing challenge results

“…Details on the framework can be found on https://github.com/wiesenfa/challengeR . We have observed that the toolkit has already been used by several users for challenge evaluation 28 , 29 and algorithm validation 30 in general. Other authors have adopted concepts from the toolkit, such as bootstrapping for investigating ranking variability 31 .…”

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

“…Some team members commented that having to dim the lights to acquire HSI was a little limiting. Further work to provide on-the-fly image calibration crucial to acquire interpretable and quantitative HSI data that can deal with changing light conditions in the OR during surgical procedures such as recently proposed in Ayala et al ( 2020 ) is therefore in progress. Additionally, we intend to conduct further in-patient intraoperative studies with a larger number of patients to fully assess and appraise the use of our iHSI device during surgery.…”

Synthetic white balancing for intra-operative hyperspectral imaging

“…The assumption of separability is intuitive from the discussion of vignetting and color constancy being addressed separately in the literature 16 , 18 , 19 , 25 , 26 . By convention, to achieve a well-posed decomposition, we constrain it such that maxi,j V(i,j)=1 and 1N∑nS(n)=1.…”

“…Spectral and spatial vignetting corrections can also be performed independently. Reference 16 proposed to use specular highlight analysis for spectral correction. This requires additional low exposure images to be obtained to prevent saturation in the specular reflection regions.…”