Composing Scalable Nonlinear Algebraic Solvers

Brune, Peter; Knepley, Matthew G.; Barry, Smith; Tu, Xuemin

doi:10.1137/130936725

Cited by 130 publications

(120 citation statements)

References 65 publications

Supporting

Mentioning

120

Contrasting

Order By: Relevance

“…In analogy with the linear case where

scriptP false(boldg; boldx false) = boldP boldg false(boldx false)

, we interpret

scriptP false(boldg; boldx false)

as the preconditioned gradient direction. By applying an optimization method with iteration

x_{k + 1} = scriptM false(boldg; x_{k} false)

scriptP false(boldg; boldx false) = boldx - scriptQ false(boldg; boldx false) = bold0

instead of to g ( x ) = 0 , we obtain the nonlinearly left‐preconditioned optimization update

{boldx}_{k + 1} = M (P (g; \cdot); {boldx}_{k}) .

This means, in practice, that all occurrences of g ( x ) in

scriptM

are replaced by

scriptP false(boldg; boldx false)

in the LP approach, as in the case of nonlinear left‐preconditioning for nonlinear equation systems . An important difference in the optimization context, however, is that we continue using the original f ( x ) and g ( x ) in determining the line search step α for methods like CG or L‐BFGS, so the gradients g ( x ) used in the line search are not replaced by

scriptP false(boldg; boldx false)

.…”

Section: Nonlinear Preconditioning Strategiesmentioning

confidence: 99%

Nonlinearly preconditioned L‐BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition

Sterck

Howse

2018

Numerical Linear Algebra App

View full text Add to dashboard Cite

Summary We derive nonlinear acceleration methods based on the limited‐memory Broyden–Fletcher–Goldfarb–Shanno (L‐BFGS) update formula for accelerating iterative optimization methods of alternating least squares (ALS) type applied to canonical polyadic and Tucker tensor decompositions. Our approach starts from linear preconditioning ideas that use linear transformations encoded by matrix multiplications and extends these ideas to the case of genuinely nonlinear preconditioning, where the preconditioning operation involves fully nonlinear transformations. As such, the ALS‐type iterations are used as fully nonlinear preconditioners for L‐BFGS, or equivalently, L‐BFGS is used as a nonlinear accelerator for ALS. Numerical results show that the resulting methods perform much better than either stand‐alone L‐BFGS or stand‐alone ALS, offering substantial improvements in terms of time to solution and robustness over state‐of‐the‐art methods for large and noisy tensor problems, including previously described acceleration methods based on nonlinear conjugate gradients and the nonlinear generalized minimal residual method. Our approach provides a general L‐BFGS‐based acceleration mechanism for nonlinear optimization.

show abstract

“…In analogy with the linear case where

scriptP false(boldg; boldx false) = boldP boldg false(boldx false)

, we interpret

scriptP false(boldg; boldx false)

as the preconditioned gradient direction. By applying an optimization method with iteration

x_{k + 1} = scriptM false(boldg; x_{k} false)

scriptP false(boldg; boldx false) = boldx - scriptQ false(boldg; boldx false) = bold0

instead of to g ( x ) = 0 , we obtain the nonlinearly left‐preconditioned optimization update

{boldx}_{k + 1} = M (P (g; \cdot); {boldx}_{k}) .

This means, in practice, that all occurrences of g ( x ) in

scriptM

are replaced by

scriptP false(boldg; boldx false)

scriptP false(boldg; boldx false)

.…”

Section: Nonlinear Preconditioning Strategiesmentioning

confidence: 99%

Nonlinearly preconditioned L‐BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition

Sterck

Howse

2018

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

“…The method based on the inverse Jacobian, Equation (22), is usually called the "bad" Broyden method because in its original applications, it did not perform as well as the "good" Broyden update, Equation (21) [66,[68][69][70][71][72]. It is more convenient to have an update to the inverse Jacobian than the Jacobian itself since (cf.…”

Section: Methodsmentioning

confidence: 99%

A Diagonally Updated Limited-Memory Quasi-Newton Method for the Weighted Density Approximation

et al. 2017

View full text Add to dashboard Cite

Abstract:We propose a limited-memory quasi-Newton method using the bad Broyden update and apply it to the nonlinear equations that must be solved to determine the effective Fermi momentum in the weighted density approximation for the exchange energy density functional. This algorithm has advantages for nonlinear systems of equations with diagonally dominant Jacobians, because it is easy to generalize the method to allow for periodic updates of the diagonal of the Jacobian. Systematic tests of the method for atoms show that one can determine the effective Fermi momentum at thousands of points in less than fifteen iterations.

show abstract

“…For problems with high nonlinearities, preconditioning techniques on the nonlinear level, such as the additive Schwarz preconditioned inexact Newton (ASPIN) methods [9,25,39], the nonlinear restricted Schwarz preconditioners [10,16], the nonlinear dual-domain decomposition methods [48], the nonlinear balancing domain decomposition by constraints methods [30], the nonlinear elimination methods [26,27,29], and the composite nonlinear algebraic methods [6], have received increasing attention in recent years. In particular, some efforts have been made in applying the ASPIN method to solve the two-phase flow problems [51,53].…”

Section: B595mentioning

confidence: 99%

“…Moreover, the subspace problem (2.6), which is a reduced-space nonlinear system, can be solved using any classical nonlinear iterative method, such as the inexact Newton method with backtracking and its many variations [7,31,46,47], and other composite solvers [6]. A key element of the NE step is the choice of S b and S g to build the subspaces V g and V b .…”

Section: Subspace Correction Phasementioning

confidence: 99%

Active-Set Reduced-Space Methods with Nonlinear Elimination for Two-Phase Flow Problems in Porous Media

Yang¹,

Yang²,

Sun³

2016

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract. Fully implicit methods are drawing more attention in scientific and engineering applications due to the allowance of large time steps in extreme-scale simulations. When using a fully implicit method to solve two-phase flow problems in porous media, one major challenge is the solution of the resultant nonlinear system at each time step. To solve such nonlinear systems, traditional nonlinear iterative methods, such as the class of the Newton methods, often fail to achieve the desired convergent rate due to the high nonlinearity of the system and/or the violation of the boundedness requirement of the saturation. In the paper, we reformulate the two-phase model as a variational inequality that naturally ensures the physical feasibility of the saturation variable. The variational inequality is then solved by an active-set reduced-space method with a nonlinear elimination preconditioner to remove the high nonlinear components that often causes the failure of the nonlinear iteration for convergence. To validate the effectiveness of the proposed method, we compare it with the classical implicit pressure-explicit saturation method for two-phase flow problems with strong heterogeneity. The numerical results show that our nonlinear solver overcomes the often severe limits on the time step associated with existing methods, results in superior convergence performance, and achieves reduction in the total computing time by more than one order of magnitude.

show abstract

Composing Scalable Nonlinear Algebraic Solvers

Cited by 130 publications

References 65 publications

Nonlinearly preconditioned L‐BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition

Nonlinearly preconditioned L‐BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition

A Diagonally Updated Limited-Memory Quasi-Newton Method for the Weighted Density Approximation

Active-Set Reduced-Space Methods with Nonlinear Elimination for Two-Phase Flow Problems in Porous Media

Contact Info

Product

Resources

About