The generalized trust region subproblem: solution complexity and convex hull results

Abstract

We consider the generalized trust region subproblem (GTRS) of minimizing a nonconvex quadratic objective over a nonconvex quadratic constraint. A lifting of this problem recasts the GTRS as minimizing a linear objective subject to two nonconvex quadratic constraints. Our first main contribution is structural: we give an explicit description of the convex hull of this nonconvex set in terms of the generalized eigenvalues of an associated matrix pencil. This result may be of interest in building relaxations for nonconvex quadratic programs. Moreover, this result allows us to reformulate the GTRS as the minimization of two convex quadratic functions in the original space. Our next set of contributions is algorithmic: we present an algorithm for solving the GTRS up to an additive error based on this reformulation. We carefully handle numerical issues that arise from inexact generalized eigenvalue and eigenvector computations and establish explicit running time guarantees for these algorithms. Notably, our algorithms run in linear (in the size of the input) time. Furthermore, our algorithm for computing an -optimal solution has a slightly-improved running time dependence on over the state-of-the-art algorithm. Our analysis shows that the dominant cost in solving the GTRS lies in solving a generalized eigenvalue problem-establishing a natural connection between these problems. Finally, generalizations of our convex hull results allow us to apply our algorithms and their theoretical guarantees directly to equality-, interval-, and hollow-constrained variants of the GTRS. This gives the first linear-time algorithm in the literature for these variants of the GTRS.

Ideal formulations for constrained convex optimization problems with indicator variables

“…Hazimeh and Mazumder [40] give specialized algorithms for the natural convex relaxation of (1) where Q is defined via strong hierarchy constraints. Several results exist concerning the convexification of nonlinear optimization problems with constraints [3,8,[15][16][17]45,49,52,[56][57][58][59], but such methods in general do not deliver ideal, compact or closed-form formulations for the specific case of problem (1) with structured feasible regions. In a recent work closely related to the setting considered here, Xie and Deng [62] prove that the perspective formulation is ideal if the objective is quadratic and separable, and Q is defined by a q-sparsity constraint.…”

Implicit regularity and linear convergence rates for the generalized trust-region subproblem

“…Generalizing the TRS, the GTRS has applications in signal processing, compressed sensing, and engineering (see [34] and references therein). The problem of minimizing a quartic of the form q(x, p(x)), where q : R n+1 → R and p : R n → R are both quadratic, can be cast in the equality-constrained variant of the GTRS.…”

“…In contrast to these papers, recent work [20,34] offers provably linear-time (in terms of the number of nonzero entries in the input data) algorithms for the GTRS using only approximate eigenvalue procedures. Jiang and Li [20] extend ideas developed in [14] for solving the TRS to derive an algorithm for solving the GTRS up to an additive error with high probability.…”

“…Here, we have elided quantities related to the condition number of the GTRS. Wang and Kılınç-Karzan [34] reexamine the convex quadratic reformulation idea of [19] and show formally that by approximating the generalized eigenvalues sufficiently well, the perturbed convex reformulation is within a small additive error of the true convex reformulation. Moreover, they establish that the resulting convex reformulation can be solved via Nesterov's accelerated gradient descent method [27,Section 2.3.3] for smooth minimax problems to achieve an overall run time guarantee of…”

The equivalence of optimal perspective formulation and Shor's SDP for quadratic programs with indicator variables

“…Sufficient conditions for exactness of SDP relaxations [e.g. 9, 23,26,30,29] and stronger rank-one conic formulations [4,5] are also given in the literature.…”