Recurrent Generative Adversarial Networks for Proximal Learning and Automated Compressive Image Recovery

Abstract

Recovering images from undersampled linear measurements typically leads to an ill-posed linear inverse problem, that asks for proper statistical priors. Building effective priors is however challenged by the low train and test overhead dictated by real-time tasks; and the need for retrieving visually "plausible" and physically "feasible" images with minimal hallucination. To cope with these challenges, we design a cascaded network architecture that unrolls the proximal gradient iterations by permeating benefits from generative residual networks (ResNet) to modeling the proximal operator. A mixture of pixel-wise and perceptual costs is then deployed to train proximals. The overall architecture resembles back-and-forth projection onto the intersection of feasible and plausible images. Extensive computational experiments are examined for a global task of reconstructing MR images of pediatric patients, and a more local task of superresolving CelebA faces, that are insightful to design efficient architectures. Our observations indicate that for MRI reconstruction, a recurrent ResNet with a single residual block effectively learns the proximal. This simple architecture appears to significantly outperform the alternative deep ResNet architecture by 2dB SNR, and the conventional compressed-sensing MRI by 4dB SNR with 100× faster inference. For image superresolution, our preliminary results indicate that modeling the denoising proximal demands deep ResNets.

Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data

“…Recently, deep learning (DL) has gained interest as a means of improving accelerated MRI reconstruction [5][6][7][8][9][10][11][12]. Several approaches have been proposed, including learning a mapping from zero-filled images to artifact-free images [10], learning interpolation rules in k-space [11,12], and a physics-based approach that utilizes the known forward model during reconstruction [6][7][8][9].…”

“…Recently, deep learning (DL) has gained interest as a means of improving accelerated MRI reconstruction [5][6][7][8][9][10][11][12]. Several approaches have been proposed, including learning a mapping from zero-filled images to artifact-free images [10], learning interpolation rules in k-space [11,12], and a physics-based approach that utilizes the known forward model during reconstruction [6][7][8][9]. The latter approach considers reconstruction as an inverse problem, including a data consistency term that involves the forward operator and a regularization term that is learned from training data.…”

Low-dose CT with deep learning regularization via proximal forward–backward splitting

“…In [20], Yang et al proposed a DL-regularized method via alternating direction method of multipliers, ADMM-net, for magnetic resonance (MR) image reconstruction. In [21], [22], Mardani et al proposed the proximal methods for MR imaging using GAN. Similar unrolling methods were also recently proposed for CT reconstruction.…”

AHP-Net: adaptive-hyper-parameter deep learning based image reconstruction method for multilevel low-dose CT

“…There are also several hyper-parameters involved in existing optimization-unrolling-based deep learning methods, which are either manually tuned-up 23,36 or are treated as one part of network weights to be learned in the training 1,27, 44 . As a result, the setting of these hyper-parameter is optimized only for one specific noise level of measurement data.…”