Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

Abstract

Purpose: Optical imaging is evolving as a key technique for advanced sensing in the operating room. Recent research has shown that machine learning algorithms can be used to address the inverse problem of converting pixel-wise multispectral reflectance measurements to underlying tissue parameters, such as oxygenation. Assessment of the specific hardware used in conjunction with such algorithms, however, has not properly addressed the possibility that the problem may be illposed. Methods: We present a novel approach to the assessment of optical imaging modalities, which is sensitive to the different types of uncertainties that may occur when inferring tissue parameters. Based on the concept of invertible neural networks, our framework goes beyond point estimates and maps each multispectral measurement to a full posterior probability distribution which is capable of representing ambiguity in the solution via multiple modes. Performance metrics for a hardware setup can then be computed from the characteristics of the posteriors. Results: Application of the assessment framework to the specific use case of camera selection for physiological parameter estimation yields the following insights: (1) Estimation of tissue oxygenation from multispectral images is a well-posed problem, while (2) blood volume fraction may not be recovered without ambiguity. (3) In general, ambiguity may be reduced by increasing the number of spectral bands in the camera. Conclusion: Our method could help to optimize optical camera design in an application-specific manner.

Explainable Deep Learning Models in Medical Image Analysis

“…Lately, uncertainty has been analyzed along with the problem of explainability in many studies to highlight the cases where a model is unsure and in turn, make the models more acceptable to non-deep learning users. There have been studies about the uncertainty of machine learning algorithms which include those for endoscopic videos [ 7 ] and tissue parameter estimation [ 8 ]. We limit the scope of this paper to explainability methods and discuss uncertainty if a study used it along with explainability.…”

Two Birds with One Stone: Investigating Invertible Neural Networks for Inverse Problems in Morphology

“…Different applications of INNs (Osborne, Armstrong, and Fletcher 2019;Ardizzone et al 2019;Adler et al 2019) have commonly used continuous latent variables with Gaussian priors as z ∼ N (μ, σ 2 ). Unlike in previously addressed illposed inverse problems, most NLP tasks contain categorical and discrete values.…”

“…For instance, inferring seismic properties of the Earth's interior from surface observations is a typical inverse problem in geophysics (Snieder and Trampert 1999); since forward problem is well-defined and can be simulated by a forward model, recovering the seismic properties that lead to a specific surface value is ill-posed. Although inverse problems have been tackled in many fields such as imaging (Bertero and Boccacci 1998;Adler et al 2019), astronomy (Osborne, Armstrong, and Fletcher 2019; Copyright © 2020, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.…”

Surgical spectral imaging

“…(2019) has proposed invertible neural networks (INNs), which aim to learn the posterior probability distribution and represent ambiguity in the solution. This has recently been applied to MSI imaging of the brain ( Adler et al., 2019 ).…”