A New Derivation of Symmetric Positive Definite Secant Updates

Schnabel, Robert B.; Dennis, John E.

doi:10.1016/b978-0-12-468662-5.50012-4

Cited by 23 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given δ k and y k , if we had all of the Cholesky factor R Q , it could be updated directly as Goldfarb [62] and Dennis and Schnabel [30]). The update for R is derived from the rank-one update to R Q implied by (6.1).…”

Section: Linearly Constrained Problemsmentioning

confidence: 99%

SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

Gill¹,

Murray²,

Saunders³

2005

SIAM Rev.

2,214

1,210

View full text Add to dashboard Cite

Sequential quadratic programming (SQP) methods have proved highly effective for solving constrained optimization problems with smooth nonlinear functions in the objective and constraints. Here we consider problems with general inequality constraints (linear and nonlinear). We assume that first derivatives are available and that the constraint gradients are sparse. Second derivatives are assumed to be unavailable or too expensive to calculate.We discuss an SQP algorithm that uses a smooth augmented Lagrangian merit function and makes explicit provision for infeasibility in the original problem and the QP subproblems. The Hessian of the Lagrangian is approximated using a limited-memory quasi-Newton method.SNOPT is a particular implementation that uses a reduced-Hessian semidefinite QP solver (SQOPT) for the QP subproblems. It is designed for problems with many thousands of constraints and variables but is best suited for problems with a moderate number of degrees of freedom (say, up to 2000). Numerical results are given for most of the CUTEr and COPS test collections (about 1020 examples of all sizes up to 40000 constraints and variables, and up to 20000 degrees of freedom). become large), SNOPT enters nonlinear elastic mode and solves the problemis called a composite objective, and the penalty parameter γ (γ ≥ 0) may take a finite sequence of increasing values. If (NP) has a feasible solution and γ is sufficiently large, the solutions to (NP) and (NP(γ)) are identical. If (NP) has no feasible solution, (NP(γ)) will tend to determine a "good" infeasible point if γ is again sufficiently large. (If γ were infinite, the nonlinear constraint violations would be minimized subject to the linear constraints and bounds.) A similar 1 formulation of (NP) is used in the SQP method of Tone [98] and is fundamental to the S 1 QP algorithm of Fletcher [38]. See also Conn [25] and Spellucci [94]. An attractive feature is that only linear terms are added to (NP), giving no increase in the expected degrees of freedom at each QP solution. 1.2. The SQP Approach. An SQP method was first suggested by Wilson [102] for the special case of convex optimization. The approach was popularized mainly by Biggs [7], Han [66], and Powell [85, 87] for general nonlinear constraints. Further history of SQP methods and extensive bibliographies are given in [61, 39, 73, 78, 28]. For a survey of recent results, see Gould and Toint [65]. Several general-purpose SQP solvers have proved reliable and efficient during the last 20 years. For example, under mild conditions the solvers NLPQL [92], NPSOL [57, 60], and DONLP [95] typically find a (local) optimum from an arbitrary starting point, and they require relatively few evaluations of the problem functions and gradients compared to traditional solvers such as MINOS [75, 76, 77] and CONOPT [34, 2].SQP methods have been particularly successful in solving the optimization problems arising in optimal trajectory calculations. For many years, the optimal trajectory system OTIS (Hargraves and Paris [67]) has been...

show abstract

Section: Linearly Constrained Problemsmentioning

confidence: 99%

SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

Gill¹,

Murray²,

Saunders³

2005

SIAM Rev.

2,214

1,210

View full text Add to dashboard Cite

show abstract

“…In this case, the Cholesky factor C k+1 of H k+1 is obtained from a rank-one change to C k (see Dennis and Schnabel [7]). We omit the details of this procedure and simply write C k+1 = update(C k , δ k , γ k ).…”

mentioning

confidence: 99%

Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization

Gill

Leonard²

2003

SIAM J. Optim.

View full text Add to dashboard Cite

Abstract. Limited-memory BFGS quasi-Newton methods approximate the Hessian matrix of second derivatives by the sum of a diagonal matrix and a fixed number of rank-one matrices. These methods are particularly effective for large problems in which the approximate Hessian cannot be stored explicitly.It can be shown that the conventional BFGS method accumulates approximate curvature in a sequence of expanding subspaces. This allows an approximate Hessian to be represented using a smaller reduced matrix that increases in dimension at each iteration. When the number of variables is large, this feature may be used to define limited-memory reduced-Hessian methods in which the dimension of the reduced Hessian is limited to save storage. Limited-memory reduced-Hessian methods have the benefit of requiring half the storage of conventional limited-memory methods.In this paper, we propose a particular reduced-Hessian method with substantial computational advantages compared to previous reduced-Hessian methods. Numerical results from a set of unconstrained problems in the CUTE test collection indicate that our implementation is competitive with the limited-memory codes L-BFGS and L-BFGS-B.

show abstract

“…It is known that operator A and its inverse can be approximated by the following recurrent formulae [19][20][21]: It is known that operator A and its inverse can be approximated by the following recurrent formulae [19][20][21]:…”

Section: Stabilized Approximate Extended Kalman Filtermentioning

confidence: 99%

“…In the same way, an auxiliary optimization task can be formulated in order to obtain an approximation of the updated estimate covariance C est kC1 defined on line 6 of Algorithm 1. The latter is guaranteed by the properties of the BFGS update formula as described in [21]. Therefore, matrix b A 1 should be at least positive semi-definite to maintain the physical properties of its exact counterpart.…”

Section: Stabilized Approximate Extended Kalman Filtermentioning

confidence: 99%

See 1 more Smart Citation

Parallel implementation of data assimilation

Bibov

Haario

2016

Numerical Methods in Fluids

View full text Add to dashboard Cite

Summary Kalman filter is a sequential estimation scheme that combines predicted and observed data to reduce the uncertainty of the next prediction. Because of its sequential nature, the algorithm cannot be efficiently implemented on modern parallel compute hardware nor can it be practically implemented on large‐scale dynamical systems because of memory issues. In this paper, we attempt to address pitfalls of the earlier low‐memory approach described in and extend it for parallel implementation. First, we describe a low‐memory method that enables one to pack covariance matrix data employed by the Kalman filter into a low‐memory form by means of certain quasi‐Newton approximation. Second, we derive parallel formulation of the filtering task, which allows to compute several filter iterations independently. Furthermore, this leads to an improvement of estimation quality as the method takes into account the cross‐correlations between consequent system states. We experimentally demonstrate this improvement by comparing the suggested algorithm with the other data assimilation methods that can benefit from parallel implementation. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

A New Derivation of Symmetric Positive Definite Secant Updates

Cited by 23 publications

References 24 publications

SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization

Parallel implementation of data assimilation

Contact Info

Product

Resources

About