Spectral organ fingerprints for intraoperative tissue classification with hyperspectral imaging

Abstract

Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, it had been unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9,059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95 %). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decision making and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

“…Our findings regarding the heterogeneity are in stark contrast to recent findings in porcine organs 40 , where the influence of the different specimen is small compared to other explanatory variables. A key difference between this study and the present work (apart from the species) is that for the porcine study all organs were healthy, whereas all of our human subjects underwent partial nephrectomy due to kidney cancer.…”

“…High inter-patient variability for kidney tissue High inter-patient variability generally suggests poor generalizability of supervised learning algorithms. In a recent porcine study we showed that the greatest source of variability related to spectral images of organs acquired from healthy animals is the organ under observation rather than the recorded individual or specific acquisition conditions 40 . This enabled us to develop a highly accurate supervised deep learning (DL) algorithm for fullyautomatic organ classification 41 .…”

Hyperspectral Imaging for the Evaluation of Microcirculatory Tissue Oxygenation and Perfusion Quality in Haemorrhagic Shock: A Porcine Study

“…Hyperspectral imaging (HSI) is an evolving noninvasive imaging technology that provides an evaluation of intrinsic biochemical tissue characteristics based on tissue-light interactions [ 4 ]. HSI technologies are increasingly being investigated in surgical research for real-time intraoperative organ perfusion-based resection planning and optimization of anastomosis quality [ 5 , 6 , 7 ]. Data on HSI technologies for microcirculatory monitoring of haemodynamic therapy in critically ill or surgical patients are limited.…”

The Advances in Computer Vision That Are Enabling More Autonomous Actions in Surgery: A Systematic Review of the Literature

“…However, unlike a traditional geometrical third dimension, spectral imaging relies on a data point derived from the reflectivity of the tissue being imaged and currently only relies on images that have two spatial dimensions. The resulting data cube is more specific than standard two-dimensional imaging and this enhanced imaging is believed to be superior at the differentiation of organs and has been utilized in space exploration and marine studies [19]. MSI measures spectral bands on a factor of 10 and HSI measures spectral bands on a scale of 100.…”

“…HSI "fingerprints" have been created in an animal model that studied the differentiation of up to 20 different internal tissues and organs with an accuracy superior to 95%. It is hoped that HSI could lead to computers ultimately being able to accurately and reliably differentiate organs in real-time [19]. The difficulty of organ identification pales into comparison of the algorithms needed to give computers the ability to differentiate the different parts of an organ, this task is known as organ segmentation.…”