Comparison of advanced large-scale minimization algorithms for the solution of inverse ill-posed problems

Alekseev, A. K.; Navon, I. M.; Steward, Jeffrey L.

doi:10.1080/10556780802370746

Cited by 38 publications

(33 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparisons of different optimization methods for data assimilation with fluid flow models are given by Alekseev and Navon [3] and Daescu and Navon [23]. QuasiNewton methods approximate the inverse of the Hessian matrix by a symmetric positive definite matrix, which is updated at every step using the new search directions and the new gradients.…”

Section: The Chemical Transport Model and The Test Casementioning

confidence: 99%

Discrete second order adjoints in atmospheric chemical transport modeling

Sandu

Zhang

2008

Journal of Computational Physics

View full text Add to dashboard Cite

Abstract. Atmospheric chemical transport models (CTMs) are essential tools for the study of air pollution, for environmental policy decisions, for the interpretation of observational data, and for producing air quality forecasts. Many air quality studies require sensitivity analyses, i.e., the computation of derivatives of the model output with respect to model parameters. The derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through adjoint sensitivity analysis. While the traditional (first order) adjoint models give the gradient of the cost functional with respect to parameters, second order adjoint models give second derivative information in the form of products between the Hessian of the cost functional and a user defined vector.In this paper we discuss the mathematical foundations of the discrete second order adjoint sensitivity method and present a complete set of computational tools for performing second order sensitivity studies in three-dimensional atmospheric CTMs. The tools include discrete second order adjoints of Runge Kutta and of Rosenbrock time stepping methods for stiff equations together with efficient implementation strategies. Numerical examples illustrate the use of these computational tools in important applications like sensitivity analysis, optimization, uncertainty quantification, and the calculation of directions of maximal error growth in three-dimensional atmospheric CTMs.

show abstract

Section: The Chemical Transport Model and The Test Casementioning

confidence: 99%

Discrete second order adjoints in atmospheric chemical transport modeling

Sandu

Zhang

2008

Journal of Computational Physics

View full text Add to dashboard Cite

show abstract

“…At the end of the backward in time integration (13) provides the gradient and the Hessian vector product…”

Section: Discrete Soamentioning

confidence: 99%

“…al. compared the performance of different minimization algorithms in the solution of inverse problems [13]. Exploring this previous work constitutes the motivation behind the following research results.…”

Section: Introductionmentioning

confidence: 99%

Second-order adjoints for solving PDE-constrained optimization problems

Cioaca

Alexe

Sandu

2012

Optimization Methods and Software

View full text Add to dashboard Cite

Inverse problems are of utmost importance in many fields of science and engineering. In the variational approach inverse problems are formulated as PDE-constrained optimization problems, where the optimal estimate of the uncertain parameters is the minimizer of a certain cost functional subject to the constraints posed by the model equations. The numerical solution of such optimization problems requires the computation of derivatives of the model output with respect to model parameters. The first order derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through first order adjoint sensitivity analysis. Second order adjoint models give second derivative information in the form of matrix-vector products between the Hessian of the cost functional and user defined vectors. Traditionally, the construction of second order derivatives for large scale models has been considered too costly. Consequently, data assimilation applications employ optimization algorithms that use only first order derivative information, like nonlinear conjugate gradients and quasi-Newton methods.In this paper we discuss the mathematical foundations of second order adjoint sensitivity analysis and show that it provides an efficient approach to obtain Hessian-vector products. We study the benefits of using of second order information in the numerical optimization process for data assimilation applications. The numerical studies are performed in a twin experiment setting with a two-dimensional shallow water model. Different scenarios are considered with different discretization approaches, observation sets, and noise levels. Optimization algorithms that employ second order derivatives are tested against widely used methods that require only first order derivatives. Conclusions are drawn regarding the potential benefits and the limitations of using high-order information in large scale data assimilation problems.

show abstract

“…Then it slows down after a relatively small number of iterations. Possibly, this is linked to the number of Hessian dominant eigenvalues [33]. From this viewpoint, the regularization is an important property of conjugate gradient algorithms [32].…”

Section: Numerical Results and Comparisonsmentioning

confidence: 99%

New accelerated conjugate gradient algorithms as a modification of Dai–Yuan’s computational scheme for unconstrained optimization

Andrei

2010

Journal of Computational and Applied Mathematics

View full text Add to dashboard Cite

a b s t r a c tNew accelerated nonlinear conjugate gradient algorithms which are mainly modifications of Dai and Yuan's for unconstrained optimization are proposed. Using the exact line search, the algorithm reduces to the Dai and Yuan conjugate gradient computational scheme. For inexact line search the algorithm satisfies the sufficient descent condition. Since the step lengths in conjugate gradient algorithms may differ from 1 by two orders of magnitude and tend to vary in a very unpredictable manner, the algorithms are equipped with an acceleration scheme able to improve the efficiency of the algorithms. Computational results for a set consisting of 750 unconstrained optimization test problems show that these new conjugate gradient algorithms substantially outperform the Dai-Yuan conjugate gradient algorithm and its hybrid variants, Hestenes-Stiefel, Polak-Ribière-Polyak, CONMIN conjugate gradient algorithms, limited quasi-Newton algorithm LBFGS and compare favorably with CG_DESCENT. In the frame of this numerical study the accelerated scaled memoryless BFGS preconditioned conjugate gradient ASCALCG algorithm proved to be more robust.

show abstract

Comparison of advanced large-scale minimization algorithms for the solution of inverse ill-posed problems

Cited by 38 publications

References 34 publications

Discrete second order adjoints in atmospheric chemical transport modeling

Discrete second order adjoints in atmospheric chemical transport modeling

Second-order adjoints for solving PDE-constrained optimization problems

New accelerated conjugate gradient algorithms as a modification of Dai–Yuan’s computational scheme for unconstrained optimization

Contact Info

Product

Resources

About