FSI Schemes: Fast Semi-Iterative Solvers for PDEs and Optimisation Methods

Hafner, David Zachary; Ochs, Peter; Weickert, Joachim; Reißel, M.; Grewenig, Sven

doi:10.1007/978-3-319-45886-1_8

Cited by 11 publications

(26 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the following, we show that an acceleration strategy of the explicit scheme induces a natural modification for the skip connections of the corresponding ResNet architecture. To speed up explicit schemes, Hafner et al [9] proposed fast semi-iterative (FSI) schemes. They perform a cycle of extrapolated explicit steps.…”

Section: Fsi Schemes and Additional Skip Connectionsmentioning

confidence: 99%

See 1 more Smart Citation

Translating Numerical Concepts for PDEs into Neural Architectures

Alt¹,

Peter²,

Weickert³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We investigate what can be learned from translating numerical algorithms into neural networks. On the numerical side, we consider explicit, accelerated explicit, and implicit schemes for a general higher order nonlinear diffusion equation in 1D, as well as linear multigrid methods. On the neural network side, we identify corresponding concepts in terms of residual networks (ResNets), recurrent networks, and U-nets. These connections guarantee Euclidean stability of specific ResNets with a transposed convolution layer structure in each block. We present three numerical justifications for skip connections: as time discretisations in explicit schemes, as extrapolation mechanisms for accelerating those methods, and as recurrent connections in fixed point solvers for implicit schemes. Last but not least, we also motivate uncommon design choices such as nonmonotone activation functions. Our findings give a numerical perspective on the success of modern neural network architectures, and they provide design criteria for stable networks.

show abstract

Section: Fsi Schemes and Additional Skip Connectionsmentioning

confidence: 99%

“…Additional interpretations of skip connections are obtained with alternative numerical methods based on fast semi-iterative (FSI) accelerations of explicit schemes [9] and on fixed point algorithms for fully implicit schemes. We show that the latter ones can be regarded as recurrent neural networks [12].…”

Section: Introductionmentioning

confidence: 99%

Translating Numerical Concepts for PDEs into Neural Architectures

Alt¹,

Peter²,

Weickert³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…(13) to evolve until a steady state has been reached, i.e., the time derivative becomes negligible. In our numerical implementation, we use a Fast Semi-Iterative Scheme (FSI) [17] to greatly accelerate convergence to a large stopping time.…”

Section: Anisotropic Edge-enhancing Fourth Order Pdementioning

confidence: 99%

“…(28). Hafner et al [17] propose a remedy to this problem, the so-called Fast Semi-Iterative Scheme (FSI). It extrapolates the basic solver iteration with the previous iterate and serves as an accelerated explicit scheme.…”

Section: Discretization and Stabilitymentioning

confidence: 99%

Fourth-Order Anisotropic Diffusion for Inpainting and Image Compression

Jumakulyyev

Schultz

2021

Mathematics and Visualization

View full text Add to dashboard Cite

Edge-enhancing diffusion (EED) can reconstruct a close approximation of an original image from a small subset of its pixels. This makes it an attractive foundation for PDE based image compression. In this work, we generalize second-order EED to a fourth-order counterpart. It involves a fourth-order diffusion tensor that is constructed from the regularized image gradient in a similar way as in traditional second-order EED, permitting diffusion along edges, while applying a non-linear diffusivity function across them. We show that our fourth-order diffusion tensor formalism provides a unifying framework for all previous anisotropic fourth-order diffusion based methods, and that it provides additional flexibility. We achieve an efficient implementation using a fast semi-iterative scheme. Experimental results on natural and medical images suggest that our novel fourth-order method produces more accurate reconstructions compared to the existing second-order EED.

show abstract

“…They drew special attention to a general class of regularized optimization problems where the accelerated PDE takes the form a damped nonlinear wave equation (generalizing (3) and (5)), and the acceleration is realized as an improvement in the CFL condition from dt ∼ dx 2 for diffusion equations (or standard gradient descent), to dt ∼ dx for wave equations. We also mention that there have been some recent approaches to acceleration in image processing, which involve solving PDEs arising from variational problems [5,17,18,40]. Since these methods are not derived from a variational (Lagrangian) perspective, the methods do not descend on an energy and lack convergence guarantees and convergence rates.…”

Section: Introductionmentioning

confidence: 99%

PDE acceleration: a convergence rate analysis and applications to obstacle problems

Calder

Yezzi

2019

Res Math Sci

View full text Add to dashboard Cite

This paper provides a rigorous convergence rate and complexity analysis for a recently introduced framework, called PDE acceleration, for solving problems in the calculus of variations, and explores applications to obstacle problems. PDE acceleration grew out of a variational interpretation of momentum methods, such as Nesterov's accelerated gradient method and Polyak's heavy ball method, that views acceleration methods as equations of motion for a generalized Lagrangian action. Its application to convex variational problems yields equations of motion in the form of a damped nonlinear wave equation rather than nonlinear diffusion arising from gradient descent. These accelerated PDE's can be efficiently solved with simple explicit finite difference schemes where acceleration is realized by an improvement in the CFL condition from dt ∼ dx 2 for diffusion equations to dt ∼ dx for wave equations. In this paper, we prove a linear convergence rate for PDE acceleration for strongly convex problems, provide a complexity analysis of the discrete scheme, and show how to optimally select the damping parameter for linear problems. We then apply PDE acceleration to solve minimal surface obstacle problems, including double obstacles with forcing, and stochastic homogenization problems with obstacles, obtaining state of the art computational results.

show abstract

FSI Schemes: Fast Semi-Iterative Solvers for PDEs and Optimisation Methods

Cited by 11 publications

References 16 publications

Translating Numerical Concepts for PDEs into Neural Architectures

Translating Numerical Concepts for PDEs into Neural Architectures

Fourth-Order Anisotropic Diffusion for Inpainting and Image Compression

PDE acceleration: a convergence rate analysis and applications to obstacle problems

Contact Info

Product

Resources

About