Variational Networks: Connecting Variational Methods and Deep Learning

Kobler, Erich; Klatzer, Teresa; Hammernik, Kerstin; Pock, Thomas

doi:10.1007/978-3-319-66709-6_23

Cited by 114 publications

(111 citation statements)

References 32 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…For example, the number of RBFs that are used to model the activation functions in a smoothed function approximation, defines the flexibility to approximate arbitrary functions in an accurate way. In our experimental setup as well as in the latest studies on image processing tasks (32), we reduced the number of RBFs compared to the initial work (6) by a half without a loss in performance but with reduced training time.…”

Section: Discussionmentioning

confidence: 99%

“…Hence, instead of adding more and more layers and creating deeper networks, we introduce more structure and flexibility in the individual layers, which might help to reduce the overall complexity of the network. As shown in (32) for image denoising and non-blind deblurring, fixing the activation functions to less flexible, e.g., convex, functions might also lead to a decrease in performance for our application.…”

Section: Discussionmentioning

confidence: 99%

“…To solve this highly non-convex training problem, we use the Inertial Incremental Proximal Gradient (IIPG) optimizer which is related to the Inertial Proximal Alternating Linearized Minimization (IPALM) algorithm (31). For algorithmic details on IIPG refer to Appendix A and (32). First-order optimizers require both the loss function value and the gradient with respect to the parameters θ .…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Learning a variational network for reconstruction of accelerated MRI data

Hammernik

Klatzer

Kobler

et al. 2017

Magnetic Resonance in Med

Self Cite

1,313

1,328

View full text Add to dashboard Cite

Variational network reconstructions preserve the natural appearance of MR images as well as pathologies that were not included in the training data set. Due to its high computational performance, that is, reconstruction time of 193 ms on a single graphics card, and the omission of parameter tuning once the network is trained, this new approach to image reconstruction can easily be integrated into clinical workflow. Magn Reson Med 79:3055-3071, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Learning a variational network for reconstruction of accelerated MRI data

Hammernik

Klatzer

Kobler

et al. 2017

Magnetic Resonance in Med

Self Cite

1,313

1,328

View full text Add to dashboard Cite

show abstract

“…Typically, the regularizer is adapted such that the end point image of the trajectory lies in a proximity of the ground truth image. However, even the general class of Field of Experts type regularizers is not able 1 arXiv:1907.08488v1 [math.OC] 19 Jul 2019 to capture the entity of the complex structure of natural images, that is why the end point image substantially differs from the ground truth image. To address this insufficient modeling, we advocate an optimal control problem using the gradient flow differential equation as the state equation and a cost functional that quantifies the distance of the ground truth image and the gradient flow trajectory evaluated at the stopping time T .…”

Section: Introductionmentioning

confidence: 99%

Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration

et al. 2020

Self Cite

View full text Add to dashboard Cite

We investigate a well-known phenomenon of variational approaches in image processing, where typically the best image quality is achieved when the gradient flow process is stopped before converging to a stationary point. This paradox originates from a tradeoff between optimization and modelling errors of the underlying variational model and holds true even if deep learning methods are used to learn highly expressive regularizers from data. In this paper, we take advantage of this paradox and introduce an optimal stopping time into the gradient flow process, which in turn is learned from data by means of an optimal control approach. As a result, we obtain highly efficient numerical schemes that achieve competitive results for image denoising and image deblurring. A nonlinear spectral analysis of the gradient of the learned regularizer gives enlightening insights about the different regularization properties.

show abstract

“…Unrolling is a generic technique to include iterative energy minimization into neural network blocks, used also in lowlevel vision [57], medical image reconstruction [28], single image depth super-resolution [44], and semantic 3D reconstruction [6]. It allows us to combine the classical multiview SR with learning-based single-view SR methods.…”

Section: Multi-view Aggregation (Mva)mentioning

confidence: 99%

Learned Multi-View Texture Super-Resolution

Richard

Cherabier

Oswald

et al. 2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object. Our architecture unifies the concepts of (i) multi-view superresolution based on the redundancy of overlapping views and (ii) single-view super-resolution based on a learned prior of high-resolution (HR) image structure. The principle of multi-view super-resolution is to invert the image formation process and recover the latent HR texture from multiple lower-resolution projections. We map that inverse problem into a block of suitably designed neural network layers, and combine it with a standard encoder-decoder network for learned single-image super-resolution. Wiring the image formation model into the network avoids having to learn perspective mapping from textures to images, and elegantly handles a varying number of input views. Experiments demonstrate that the combination of multi-view observations and learned prior yields improved texture maps.

show abstract

Variational Networks: Connecting Variational Methods and Deep Learning

Cited by 114 publications

References 32 publications

Learning a variational network for reconstruction of accelerated MRI data

Learning a variational network for reconstruction of accelerated MRI data

Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration

Learned Multi-View Texture Super-Resolution

Contact Info

Product

Resources

About