Spectral estimates for unreduced symmetric KKT systems arising from Interior Point methods

Abstract

We consider symmetrized Karush-Kuhn-Tucker systems arising in the solution of convex quadratic programming problems in standard form by Interior Point methods. Their coefficient matrices usually have 3 3 block structure, and under suitable conditions on both the quadratic programming problem and the solution, they are nonsingular in the limit. We present new spectral estimates for these matrices: the new bounds are established for the unpreconditioned matrices and for the matrices preconditioned by symmetric positive definite augmented preconditioners. Some of the obtained results complete the analysis recently given by ‡ In the context of IP methods, the coefficient matrix of the system to be solved at each iteration is often referred to as the barrier KKT matrix. Following [7, p. 92] we omit the term "barrier" for simplicity and denote the system as KKT system, see also [8,9].B. MORINI, V. SIMONCINI AND M. TANIwhere J 2 R m n has full row rank m < n, H 2 R n n is symmetric and positive semidefinite (SPSD in the following), x;´; c 2 R n , y; b 2 R m , and the inequalities are meant componentwise. The application of a primal-dual IP method gives rise, at each iteration, to an unsymmetric 3 3 block matrix of dimension 2n C m, which allows for alternative formulations of differing dimension, conditioning and definiteness [3,[9][10][11]. In fact, the unsymmetric 3 3 matrix can be easily symmetrized without increasing the conditioning of the system [7], and here, we will refer to the resulting symmetric matrix as the unreduced KKT matrix. On the other hand, by exploiting the structure of the unsymmetric 3 3 matrix and block elimination, it is common to use a linear system of dimension n C m with a reduced (or augmented) symmetric 2 2 KKT matrix. Finally, a further block elimination may yield a condensed system (or normal equations) where the matrix is a Schur complement of dimension n. The focus of this work is the theoretical study of the unreduced 3 3 formulation and the numerical illustration of the obtained results. Unlike the reduced 2 2 matrix, under suitable conditions on both the problem (1) and the solution, the unreduced KKT matrix has condition number asymptotically uniformly bounded and typically remains well-conditioned as the solution is approached [3,7]. Motivated by this feature, in a very recent paper, Greif et al. [12] presented a spectral analysis for the 3 3 matrix and claimed that this formulation can be preferable to the reduced one in terms of eigenvalues and conditioning, although a 'benign' asymptotically ill-conditioned scaling has to be applied to the right-hand side and to the system variables [3]. Such a study covers also variants of KKT matrices arising from regularization of the optimization problem.The study conducted in [12] has renewed the interest in the unreduced formulation but leaves some issues open that are worth investigating and are a prerequisite for a thorough comparison of the 2 2 and 3 3 formulations. Specifically, some eigenvalue bounds presented in [12] may be overly pe...

Spectral estimates for saddle point matrices arising in weak constraint four‐dimensional variational data assimilation

“…Such systems are explored in the optimization literature. 20,21,29 When solving these systems iteratively, it is usually assumed that calculations involving the blocks on the diagonal are computationally expensive, while the off-diagonal blocks are cheap to apply and easily approximated. However, in our application, operations with the diagonal blocks are relatively cheap and the off-diagonal blocks are computationally expensive, particularly because of the integrations of the model and its adjoint in L and L T .…”

“…Since D− I is positive definite, from (28) u = −(D− I) −1 Lv. Using this expression and multiplying (29)…”

Uzawa methods for a class of block three‐by‐three saddle‐point problems

“…In addition, we emphasize that the saddle‐point problem , in essence, is quite different from the block three‐by‐three linear systems considered in other works . Actually, the latter one possesses the form of where X and Z are diagonal matrices and positive definite, and I is the identity matrix.…”

“…In addition, we emphasize that the saddle-point problem (1), in essence, is quite different from the block three-by-three linear systems considered in other works. [33][34][35][36] Actually, the latter one possesses the form of…”

A Piecewise Bound Constrained Optimization for Harmonic Responsibilities Assessment under Utility Harmonic Impedance Changes

“…The interior-point [27] for constrained optimization is mainly used to solve a variety of approximate optimization problem. For each η > 0 the approximate model can be expressed as: where s i denotes the slack variable.…”