Minimizing convex quadratics with variable precision conjugate gradients

Abstract

We investigate the method of conjugate gradients, exploiting inaccurate matrix-vector products, for the solution of convex quadratic optimization problems. Theoretical performance bounds are derived, and the necessary quantities occurring in the theoretical bounds estimated, leading to a practical algorithm. Numerical experiments suggest that this approach has significant potential, including in the steadily more important context of multiprecision computations.

Analysis of the Truncated Conjugate Gradient Method for Linear Matrix Equations

“…More precisely, preserved local orthogonality seems to be sufficient for the method to advance the approximation with linear convergence -we expect superlinear convergence to be lost, as it occurs in finite precision CG; see, e.g., [24] and references therein. The importance of local orthogonality has been stressed in the past to enhance convergence properties of inexact preconditioned CG, see, e.g., [8], [26], [9] and their references. Similar pictures can also be observed in analyzing round-off effects in the GMRES orthonormal basis -constructed with the modified Gram-Schmidt algorithm, in which the residual norm stagnates at the level where all linear independence is lost [11].…”

Exploiting variable precision in GMRES

“…If no more than Kmax iterations are to be performed, we can let φj = K −1/2 max (although more elaborate choices for φj could be considered; see for example [8]).…”

“…There is already a literature on the use of inexact matrix-vector products in GM-RES and other Krylov subspace methods; see, e.g., [19,6,3,7,8] and the references therein. This work is not a simple extension of such results.…”

Inexact Restoration for Minimization with Inexact Evaluation both of the Objective Function and the Constraints

“…Moreover, in [13] the case in which derivatives are not available was considered. Inexactness of the objective function in optimization problems was addressed in several additional papers in recent years [7,8,9,34,35,39,33]. The objective of the present paper is to use the ideas of [40,11,12] to handle the constrained optimization problem in which the evaluation of the objective functions and the constraints is subject to error.…”