Convolutional proximal neural networks and Plug-and-Play algorithms

Hertrich, Johannes; Neumayer, Sebastian; Steidl, Gabriele

doi:10.1016/j.laa.2021.09.004

Cited by 40 publications

(31 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, we set the reconstruction to be x = x 100 . It was shown in [3,15,19,33,35,51] that Plug-and-Play methods can achieve state-of-the-art performance for several applications.…”

Section: Comparison With Established Methodsmentioning

confidence: 99%

See 1 more Smart Citation

WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution

Altekrüger¹,

Hertrich²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce WPPNets, which are CNNs trained by a new unsupervised loss function for image superresolution of materials microstructures. Instead of requiring access to a large database of registered high-and low-resolution images, we only assume to know a large database of low resolution images, the forward operator and one high-resolution reference image. Then, we propose a loss function based on the Wasserstein patch prior which measures the Wasserstein-2 distance between the patch distributions of the predictions and the reference image. We demonstrate by numerical examples that WPPNets outperform other methods with similar assumptions. In particular, we show that WPPNets are much more stable under inaccurate knowledge or perturbations of the forward operator. This enables us to use them in real-world applications, where neither a large database of registered data nor the exact forward operator are given.

show abstract

“…Then, we set the reconstruction to be x = x 100 . It was shown in [3,15,19,33,35,51] that Plug-and-Play methods can achieve state-of-the-art performance for several applications.…”

Section: Comparison With Established Methodsmentioning

confidence: 99%

“…Plug-and-Play methods were used for several applications with excellent performance, see e.g. [3,15,19,33,35,51]. Closely related to plug-andplay methods are regularizing by denoising (RED) [37], variational networks [13] and total deep variation [26].…”

Section: Introductionmentioning

confidence: 99%

WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution

Altekrüger¹,

Hertrich²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…They are also more versatile as the CNN denoisers can be used for different kinds of inverse problems without the need for retraining. The difficulty in employing these learning-based iterative schemes is that the Lipschitz constant of the CNNs must be controlled in order to ensure their convergence [46], [47], which is not straightforward and remains an active area of research [47]- [49].…”

Section: Deep Learning Based Methodsmentioning

confidence: 99%

Bayesian Inversion for Nonlinear Imaging Models using Deep Generative Priors

Bohra¹,

Pham²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

Most modern imaging systems involve a computational reconstruction pipeline to infer the image of interest from acquired measurements. The Bayesian reconstruction framework relies on the characterization of the posterior distribution, which depends on a model of the imaging system and prior knowledge on the image, for solving such inverse problems. Here, the choice of the prior distribution is critical for obtaining highquality estimates. In this work, we use deep generative models to represent the prior distribution. We develop a posterior sampling scheme for the class of nonlinear inverse problems where the forward model has a neural-network-like structure. This class includes most existing imaging modalities. We introduce the notion of augmented generative models in order to suitably handle quantitative image recovery. We illustrate the advantages of our framework by applying it to two nonlinear imaging modalities-phase retrieval and optical diffraction tomography.

show abstract

“…radio images of high resolution and dynamic range) requiring a large number of iterations to reach convergence. Several methods have been proposed to ensure the firm nonexpansiveness of the denoiser (Ryu et al 2019;Terris et al 2020;Hertrich et al 2021), yet often coming at the cost of either strong architectural constraints, or inaccurate control of the nonexpansiveness. In our recent work Pesquet et al (2021), we proposed to augment the denoiser's training loss with a firm nonexpansiveness term, in order to meet the PnP convergence conditions with no restrictions on the DNN architecture.…”

Section: Proposed Hybrid Plug-and-play Frameworkmentioning

confidence: 99%

“…Several works have recently focused on restoring the convergence of PnP algorithms using DNN denoisers. The vast majority of these works ensure a nonexpansiveness constraint on the denoiser (Romano et al 2017;Ryu et al 2019;Cohen et al 2021;Hertrich et al 2021), but this constraint often comes with either restrictive assumptions on the algorithm, on the DNN architecture or on the operator ∇ 𝑓 .…”

Section: Pnp-fbmentioning

confidence: 99%

Image reconstruction algorithms in radio interferometry: from handcrafted to learned regularization denoisers

Terris¹,

Dabbech²,

Tang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a new class of iterative image reconstruction algorithms for radio interferometry, at the interface of convex optimization and deep learning, inspired by plug-and-play methods. The approach consists in learning a prior image model by training a deep neural network (DNN) as a denoiser, and substituting it for the handcrafted proximal regularization operator of an optimization algorithm. The proposed AIRI ("AI for Regularization in Radio-Interferometric Imaging") framework, for imaging complex intensity structure with diffuse and faint emission, inherits the robustness and interpretability of optimization, and the learning power and speed of networks. Our approach relies on three steps. Firstly, we design a low dynamic range database for supervised training from optical intensity images. Secondly, we train a DNN denoiser with basic architecture ensuring positivity of the output image, at a noise level inferred from the signal-to-noise ratio of the data. We use either ℓ 2 or ℓ 1 training losses, enhanced with a nonexpansiveness term ensuring algorithm convergence, and including on-the-fly database dynamic range enhancement via exponentiation. Thirdly, we plug the learned denoiser into the forward-backward optimization algorithm, resulting in a simple iterative structure alternating a denoising step with a gradient-descent data-fidelity step. The resulting AIRI-ℓ 2 and AIRI-ℓ 1 were validated against CLEAN and optimization algorithms of the SARA family, propelled by the "average sparsity" proximal regularization operator. Simulation results show that these first AIRI incarnations are competitive in imaging quality with SARA and its unconstrained forward-backward-based version uSARA, while providing significant acceleration. CLEAN remains faster but offers lower reconstruction quality.

show abstract

Convolutional proximal neural networks and Plug-and-Play algorithms

Cited by 40 publications

References 48 publications

WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution

WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution

Bayesian Inversion for Nonlinear Imaging Models using Deep Generative Priors

Image reconstruction algorithms in radio interferometry: from handcrafted to learned regularization denoisers

Contact Info

Product

Resources

About