Towards machine learning for heterogeneous inverse scattering in 3D microscopy

Abstract

Light propagating through a nonuniform medium scatters as it interacts with particles with different refractive properties such as cells in the tissue. In this work we aim to utilize this scattering process to learn a volumetric reconstruction of scattering parameters, in particular particle densities. We target microscopy applications where coherent speckle effects are an integral part of the imaging process. We argue that the key for successful learning is modeling realistic speckles in the training process. To this end, we build on the development of recent physically accurate speckle simulators. We also explore how to incorporate speckle statistics, such as the memory effect, in the learning framework. Overall, this paper contributes an analysis of multiple aspects of the network design including the learning architecture, the training data and the desired input features. We hope this study will pave the road for future design of learning based imaging systems in this challenging domain.

A novel multi-frame wavelet generative adversarial network for scattering reconstruction of structured illumination microscopy

“…We used the open source BioSR biostructure dataset (Qiao and Li 2022) to create the training and test datasets. The dataset employed a multimodal SIM system to acquire a dataset of clathrin-coated pits (CCPs), endoplasmic reticulum (ER), microtubules (MTs) and F-actin filaments (Wertheimer et al 2022), which includes 2000 pairs of blurred and clear image sequences. Each type of sample has an average of 50 distinct regions-ofinterest.…”

“…However, the scattering medium of biological tissues, especially complex biological structures, can cause obstacles to image formation (Malavalli and Aegerter 2019), resulting in a lower resolution of the obtained images. This is because during SIM imaging, the propagation of fluorescence in a non-uniform medium interacts with cells in the tissue, resulting in low-resolution scattered images due to the different refractive properties of cells in the tissue (Wertheimer et al 2022). Currently, the latest hardware-improved multimodal SIM systems (Qiao et al 2021b) can clearly observe the dynamic processes of cells and have become a fast-growing need to solve the above problems.…”

Variable Imaging Projection Cloud Scattering Tomography

“…Recently, various works utilized machine learning to assess atmospheric radiative properties, using single-view image data [35], [36], [37]. Wertheimer et al [38] suggested a DNN for recovering a heterogeneous scattering volume having a few degrees of freedom, using coherent lighting. The closest work to ours is that of Sde-Chen et al [39], whose DNN-based system (3DeepCT) performs CT of clouds, having tens of thousands of unknown voxels' values, using scattering of incoherent light.…”