Conditional Injective Flows for Bayesian Imaging

Abstract

Deep learning is the current de facto state of the art in tomographic imaging. A common approach is to feed the result of a simple inversion, for example the backprojection, to a convolutional neural network (CNN) which then computes the reconstruction. Despite strong results on "in-distribution" test data similar to the training data, backprojection from sparse-view data delocalizes singularities, so these approaches require a large receptive field to perform well. As a consequence, they overfit to certain global structures which leads to poor generalization on out-of-distribution (OOD) samples. Moreover, their memory complexity and training time scale unfavorably with image resolution, making them impractical for application at realistic clinical resolutions, especially in 3D: a standard U-Net requires a substantial 140GB of memory and 2600 seconds per epoch on a research-grade GPU when training on 1024 × 1024 images. In this paper, we introduce GLIMPSE, a local processing neural network for computed tomography which reconstructs a pixel value by feeding only the measurements associated with the neighborhood of the pixel to a simple MLP. While achieving comparable or better performance with successful CNNs like the U-Net on in-distribution test data, GLIMPSE significantly outperforms them on OOD samples while maintaining a memory footprint almost independent of image resolution; 5GB memory suffices to train on 1024 × 1024 images. Further, we built GLIMPSE to be fully differentiable, which enables feats such as recovery of accurate projection angles if they are out of calibration.

Deep Injective Prior for Inverse Scattering

“…We assess the performance of the proposed methods for MAP estimation and posterior modeling on synthetic and experimental data. We train the model on two synthetic largescale datasets: 1) MNIST [64] with 60000 training samples in the resolution N = 32, and 2) a more challenging dataset we generated comprising 60000 training samples with resolution N = 64 of overlapping ellipses used in [14]. Figure 5 shows example test contrasts, their projections on the learned manifold, and the samples generated by the injective network, verifying the ability of the model to produce outputs of good quality.…”

“…Most deep learning models employed for inverse scattering adopt a supervised learning approach, which trains a deep neural network to regress the permittivity pattern. Some studies [12]- [14] have utilized scattered fields as the input of the neural network. Despite the satisfactory reconstructions [14], these methods are sensitive to changes in the experimental configuration, such as frequency, the number of transmitters and receivers or other real-world factors.…”

“…Some studies [12]- [14] have utilized scattered fields as the input of the neural network. Despite the satisfactory reconstructions [14], these methods are sensitive to changes in the experimental configuration, such as frequency, the number of transmitters and receivers or other real-world factors. Even slight variations in the distribution of scattered fields in test time can lead to a significant degradation in reconstruction quality, requiring costly acquisition of new training data.…”

“…Back-projections can be used as input to tackle some of these issues [15]- [17]. While this approach yields good reconstructions for objects with small and moderate permittivity, due to the non-linearity the quality of back-projections significantly drops in large permittivity leading to a drop in the reconstruction quality [14]. Moreover, supervised learning methods are vulnerable to adversarial attacks [18], which is problematic in medical applications [19].…”

“…A point estimate, however, is often insufficient or misleading due to the ill-posedness of the inverse scattering problem. This limitation can be tackled by applying Bayesian frameworks based on deep learning networks [14], [51], [52] to generate multiple estimates of the permittivity and perform uncertainty quantification (UQ). However, these methods are supervised and suffer from the aforementioned issues.…”