Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network

Abstract

Coherent imaging through scatter is a challenging topic in computational imaging. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach for coherent imaging through scatter can make high-quality predictions through unseen diffusers. Here, we propose a new deep neural network (DNN) model that is agnostic to a broader class of perturbations including scatter change, displacements, and system defocus up to 10X depth of field. In addition, we develop a new analysis framework for interpreting the mechanism of our DNN model and visualizing its generalizability based on an unsupervised dimension reduction technique. We show that the DNN can unmix the diffuser/displacement information and distill the objectspecific information to achieve generalization under different scattering conditions. Our work paves the way to a highly scalable deep learning approach to different scattering conditions and a new framework for network interpretation.

Image Transmission Through a Dynamically Perturbed Multimode Fiber by Deep Learning

“…In ref. [30], Li et al. used an unsupervised dimension reduction technique and demonstrated that speckle that emerge from thin diffusers can be clustered according to their original class or acquisition configuration.…”

“…trained a CNN on the speckle created by a group of thin diffusers, and produced excellent image reconstructions from speckle resulting from different diffusers of the same type. [ 20,30 ] This generalization was possible due to the existence of correlations between speckle created by the different diffusers, an invariant property which the DL model learned to recognize.…”

Adaptive 3D descattering with a dynamic synthesis network

“…Deep learning (DL) has become a powerful technique for tackling complex yet important computational imaging problems [1], such as phase imaging [2][3][4][5], tomography [6][7][8][9], ghost imaging [10][11][12][13], lightfield microscopy [14,15], super-resolution imaging [16][17][18], digital holography [19][20][21], and imaging through scattering media [22][23][24][25]. Within these computational imaging applications, one of the prevalent problems is "descattering", or removing scattering artifacts.…”

“…While increasingly effective, the existing descattering DL frameworks are fundamentally impeded by an outstanding challenge. Specifically, they generally demonstrate optimal performance only when the scattering condition in the testing data match well with the training data, and the performance sharply degrades when the scattering conditions are mismatched [22,24]. Thus, if a task requires working with many different levels of scattering, it generally needs to train multiple "expert" networks, each optimized for a specific scattering condition [9,13,26].…”

“…Such expert networks are very limiting since they require a priori knowledge of the scattering level. An alternative method is to train a single "generalist" network using data from many different scattering conditions [22,24]. However, this generalist approach often leads to degraded performance as compared to the expert [9,13,26].…”