Out of Distribution Detection for Intra-operative Functional Imaging

Adler, Tim; Ayala, Leonardo; Ardizzone, Lynton; Kenngott, Hannes; Vemuri, Anant; Müller‐Stich, Beat P.; Rother, Carsten; Köthe, Ullrich; Maier-Hein, Lena

doi:10.1007/978-3-030-32689-0_8

“…In the field of multispectral imaging, OOD metrics were used to quantify domain gaps between data that a deep learning algorithm was trained on and newly acquired experimental data. 155,156 We expect the investigation of well-calibrated methods for uncertainty estimation and the automatic detection of OOD scenarios to be integral towards the clinical translation of deep learning-based PAI methodologies.…”

Section: Discussionmentioning

confidence: 99%

Deep learning for biomedical photoacoustic imaging: A review

Gröhl,

Schellenberg,

Dreher

et al. 2020

Preprint

Self Cite

0

View full text Add to dashboard Cite

Photoacoustic Imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.

show abstract

“…The INNs were implemented using the PyTorch framework and the FrEIA package for invertible architectures. Following up on our previous work with INNs 59, 61, 68 , we applied the normalizing flow architecture originally introduced in Dinh et al 60 and used the following network default settings: 20 affine coupling blocks 60 with 3 layer fully-connected subnetworks with 256 hidden dimensions, rectified linear unit (ReLU) activations and fixed channel-permutations. The networks were trained using Maximum-Likelihood training, i.e.…”

Section: Methodsmentioning

confidence: 99%

“…Note that we presented the general idea of using INNs for OoD in the context of uncertainty quantification at a conference workshop 61 . However, the idea of phrasing the ischemia detection problem as an OoD problem is entirely novel (patent pending).…”

Section: Ischemia Indexmentioning

confidence: 99%

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Ayala

¹

,

Adler

²

,

Seidlitz

³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. Spectral imaging takes advantage of the fact that different tissue components have unique optical properties to recover relevant information on tissue function such as ischemia. However, clinical success stories for advancing laparoscopic surgery with spectral imaging are lacking to date. To address this bottleneck, we developed the first laparoscopic real-time multispectral imaging (MSI) system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional RGB (Red, Green, and Blue) surgical view of the patient with functional information at a video rate of 25 Hz. To account for the high inter-patient variability of human tissue, we phrase the problem of ischemia detection as an out-of-distribution (OoD) detection problem that does not rely on data from any other patient. Using an ensemble of invertible neural networks (INNs) as a core component, our algorithm computes the likelihood of ischemia based on a short (several seconds) video sequence acquired at the beginning of each surgery. A first-in-human trial performed on 10 patients undergoing partial nephrectomy demonstrates the feasibility of our approach for fully-automatic live ischemia monitoring during laparoscopic surgery. Compared to the clinical state-of-the-art approach based on indocyanine green (ICG) fluorescence, the proposed MSI-based method does not require the injection of a contrast agent and is repeatable if the wrong segment has been clamped. Spectral imaging combined with advanced deep learning-based analysis tools could thus evolve as an important tool for fast, efficient, reliable and safe functional imaging in minimally invasive surgery.

show abstract

“…Note that we presented the general idea of using INNs for OoD in the context of uncertainty quantification at a conference workshop (49). However, the idea of phrasing the ischemia detection problem as an OoD problem is entirely novel (patent pending).…”

Section: Ischemia Indexmentioning

confidence: 99%

Spectral imaging enables contrast agent–free real-time ischemia monitoring in laparoscopic surgery

Ayala

¹

,

Adler

²

,

Seidlitz

³

et al. 2023

View full text Add to dashboard Cite

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent–free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning–based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

show abstract

Out of Distribution Detection for Intra-operative Functional Imaging

Cited by 6 publications

References 8 publications

Deep learning for biomedical photoacoustic imaging: A review

Deep learning for biomedical photoacoustic imaging: A review

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Spectral imaging enables contrast agent–free real-time ischemia monitoring in laparoscopic surgery

Contact Info

Product

Resources

About