Preconditioners for Indefinite Systems Arising in Optimization

Abstract

For any indefinite matrix K, it is shown that the UTDU factorization can be modified at nominal cost to provide an "exact" preconditioner for SYMMLQ. Code is given for overwriting the blockdiagonal matrix D produced by MA27.The KKT systems arising in barrier methods for linear and nonlinear programming are studied, and preconditioners for use with SYMMLQ are derived.For nonlinear programs a preconditioner is derived from the "smaller" KKT system associated with variables that are not near a bound. For linear programs several preconditioners are proposed, based on a square nonsingular matrix B that is analogous to the basis matrix in the simplex method.The aim is to facilitate solution of full KKT systems rather than equations of the form AD2ATAr r when the latter become excessively ill conditioned.

Preconditioning Indefinite Systems in Interior Point Methods for Optimization

“…Such preconditioners are widely used in linear systems obtained from a discretization of partial differential equations [15,24]. The preconditioners for the augmented system have also been used in the context of linear programming [14,23] and in the context of nonlinear programming [11,19,21,22,25]. As was shown in [23], the preconditioners for indefinite augmented system offer more freedom than those for the normal equations.…”

“…The preconditioner needs to involve matrix −1 to capture the key numerical properties of (14). However, we need to make it significantly less expensive than the direct factorization of this matrix so we drop out all off-diagonal elements in the matrix Q.…”

“…Matrix P of (3) belongs to a wide class of block-preconditioners that have been studied in numerous applications in the context of partial differential equations, see for example [15,24] and the references therein, and in the context of optimization [11,14,19,21,22]. We perform the spectral analysis of P −1 H and conclude, as in [19], that for convex optimization problems all the eigenvalues of this matrix are strictly positive.…”

Sparse block factorization of saddle point matrices

“…Numerous solution methods for the saddle point systems of the form (1) can be found in the literature and many of them have focused on preconditioning techniques for Krylov subspace iterative solvers [1,2,3,7,12,18,23,24,26,28,31]. As a direct method against iterative solvers, various techniques on symmetric indefinite factorization P TÅ P = LDL T can be found in [9,14,21,34,35,38], where P is a permutation matrix, L is unit lower triangular matrix, D is block-diagonal matrix with blocks of order 1 or 2.…”

“…Systems of the form (1) are known as saddle point problems, which are resulted from discretization of PDEs or coupled PDEs such as the Stokes and mixed finite element methods. Saddle point systems also arise in electronic circuit simulations [30,35], Maxwell's equations [26], economic models and constrained optimization problems [1,10,15,16,18,33]. For example consider the equality-constrained quadratic programming problem: min x f (x) = 1 2x TÅx −x Tf subject toBx =g.…”