Spectral Organ Fingerprints For Machine Learning-Based Intraoperative Tissue Classification With Hyperspectral Imaging

Abstract

Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, it had been unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9,059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95 %). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decision making and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.Category for submissionExperimental studyLevel of EvidenceNot applicable. Translational animal science. Original article.

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

“…The tissue segmentation appears to be the third largest studied task in this survey comprising around 12% of the total articles (see Table 6). This section comprises all the studies performed on tissues including vessel segmentation, edge detection, healthy and cancerous tissue classification, uncertainty inference segmentation, and tissue retraction [164]- [175].…”

“…The applicability of tissue segmentation spans the liver to brain to kidney to lungs and several other organs [175]- [182]. It also includes the binary classification of healthy and cancerous tissue [170], [172], [183].…”