Preconditioning Indefinite Systems in Interior Point Methods for Optimization

Abstract

Abstract. Every Newton step in an interior-point method for optimization requires a solution of a symmetric indefinite system of linear equations. Most of today's codes apply direct solution methods to perform this task. The use of logarithmic barriers in interior point methods causes unavoidable ill-conditioning of linear systems and, hence, iterative methods fail to provide sufficient accuracy unless appropriately preconditioned. Two types of preconditioners which use some form of incomplete Cholesky factorization for indefinite systems are proposed in this paper. Although they involve significantly sparser factorizations than those used in direct approaches they still capture most of the numerical properties of the preconditioned system. The spectral analysis of the preconditioned matrix is performed: for convex optimization problems all the eigenvalues of this matrix are strictly positive. Numerical results are given for a set of public domain large linearly constrained convex quadratic programming problems with sizes reaching tens of thousands of variables. The analysis of these results reveals that the solution times for such problems on a modern PC are measured in minutes when direct methods are used and drop to seconds when iterative methods with appropriate preconditioners are used.

Inexact constraint preconditioners for linear systems arising in interior point methods

“…We extend the analysis of [4] to this case and provide a complete characterisation of the spectrum ofP −1 H. Our findings can be summarised as follows. Suppose both A andà have full rank m and the error of approximation E = A −à has rank p, where 0 ≤ p ≤ m. The use of approximation of A allows for a presence of complex eigenvalues in the preconditioned matrix.…”

“…The first requirement helps to limit the number of complex eigenvalues and the second helps to keep the bound on |λ − 1| small. Following the arguments in [4], after dropping all off-diagonal elements from Q and using sparserà we expect an important gain in the sparsity of the Cholesky-like factor of preconditioner (1.3). Moreover, as in [4], we can exploit the diagonal form of D when computing the inverse representation ofP .…”

“…Following the arguments in [4], after dropping all off-diagonal elements from Q and using sparserà we expect an important gain in the sparsity of the Cholesky-like factor of preconditioner (1.3). Moreover, as in [4], we can exploit the diagonal form of D when computing the inverse representation ofP . Indeed, we open the possibility of reducing the augmented system to the normal equations form whenever the latter offers any advantages.…”

“…This is because certain large instances of (1.1) defy direct methods (the inverse representation of the matrix involved requires prohibitive memory resources and cannot be computed efficiently). A variety of preconditioners have been proposed for such matrices, notably [4,9,10,11] to mention a few. They have a common feature of constructing the approximation to (1.1) by simplifying its upper left block, namely by applying the preconditioner of the form…”

“…The latter situation has been studied for example in [4]: this choice has an advantage because it allows for further reduction of the preconditioner P to the form of normal equations (Schur complement) where a reduced system of form AD −1 A T is computed. The Hessian matrix Q cannot be approximated by anything simpler than a diagonal matrix.…”