Tomographic reconstruction with a generative adversarial network

Abstract

This paper presents a deep learning algorithm for tomographic reconstruction (GANrec). The algorithm uses a generative adversarial network (GAN) to solve the inverse of the Radon transform directly. It works for independent sinograms without additional training steps. The GAN has been developed to fit the input sinogram with the model sinogram generated from the predicted reconstruction. Good quality reconstructions can be obtained during the minimization of the fitting errors. The reconstruction is a self‐training procedure based on the physics model, instead of on training data. The algorithm showed significant improvements in the reconstruction accuracy, especially for missing‐wedge tomography acquired at less than 180° rotational range. It was also validated by reconstructing a missing‐wedge X‐ray ptychographic tomography (PXCT) data set of a macroporous zeolite particle, for which only 51 projections over 70° could be collected. The GANrec recovered the 3D pore structure with reasonable quality for further analysis. This reconstruction concept can work universally for most of the ill‐posed inverse problems if the forward model is well defined, such as phase retrieval of in‐line phase‐contrast imaging.

Limited-angle computed tomography with deep image and physics priors

“…GridRec is demonstrated as the primal reconstruction method that does not take any steps to compensate for missing wedge artifacts or noise. A more elaborate and advanced algorithm is developed by Yang et al 22 , deploying a self-training approach with a Generative Adversarial Network (GAN), called GANrec, and aiming at accurate slice-by-slice reconstructions by the use of a generative network with the help of a discriminator network loss. Although the method is successful for reconstructions in noise-free cases and eliminates some of the missing wedge artifacts, it is not robust to noisy projection inputs.…”

“…In a related study, DIP is used for the limited angle problem but for diffraction tomography 21 , where the measured images are in the Fourier space. In parallel, Yang et al 22 introduced a similar self-training optimization approach by combining DIP and tomographic reconstruction through a single network. While they showed great improvements compared to non-learning approaches, they also reported the instability of the method when measurements are noisy.…”

“…Because with ADMM, we can break the imaging problem into smaller and well-defined sub-problems that can be solved independently, we can effectively map those sub-problems onto available computing resources. To further improve the method, we extend the work of Yang et al 22 by incorporating a fully-connected network before the DIP network, as the reconstruction is heavily dependent on the inverse projection operator, and a fully connected network can be trained as one. Through numerical experiments, we analyze the convergence behavior of the method and show that our approach can outperform standard methods like FBP and TV, based on the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).…”

Strategies of Deep Learning for Tomographic Reconstruction

“…The robustness of the CNN classifier in evaluating the features of the image has been proved in practice. To avoid the 'fake convergence' in the model-based learning with the cost function, we developed an image reconstruction based on GAN (GANrec) [17]. GANrec modified the conditional GAN [18] for the iterative image reconstruction.…”

PtyNAMi: ptychographic nano-analytical microscope

“…It should be noted that the MEMS chips can in fact be used directly for complementary TEM imaging, if the sample size permits. In addition, limited-angle rotation has shown potential for the acquisition of ptycho-tomographic data under in situ conditions, with the potential to examine 3D structural changes in greater detail (Yang et al, 2020).…”