Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints

Abstract

We propose a framework for modeling and solving low-rank optimization problems to certifiable optimality.We introduce symmetric projection matrices that satisfy Y 2 = Y , the matrix analog of binary variables that satisfy z 2 = z, to model rank constraints. By leveraging regularization and strong duality, we prove that this modeling paradigm yields tractable convex optimization problems over the non-convex set of orthogonal projection matrices. Furthermore, we design outer-approximation algorithms to solve low-rank problems to certifiable optimality, compute lower bounds via their semidefinite relaxations, and provide near optimal solutions through rounding and local search techniques. We implement these numerical ingredients and, for the first time, solve low-rank optimization problems to certifiable optimality. Our algorithms also supply certifiably near-optimal solutions for larger problem sizes and outperform existing heuristics, by deriving an alternative to the popular nuclear norm relaxation which generalizes the perspective relaxation from vectors to matrices. All in all, our framework, which we name Mixed-Projection Conic Optimization, solves low-rank problems to certifiable optimality in a tractable and unified fashion.

On the convex hull of convex quadratic optimization problems with indicators

“…In the proof of Proposition 1, with the assumption Q ≻ 0, constraints W W † x = x enforce the complementarity constraints x • (e − z) = 0, and therefore, such constraints are excluded in (3). In contrast, in the proof of Proposition 4, with Q potentially of low-rank, constraints W W † F ⊤ x = F ⊤ x alone are not sufficient to enforce x • (e − z) = 0, and therefore, they are included in (10) and are used to prove the validity of the mixed-integer formulation. Indeed, note that if there exist S ∈ Z and x ∈ R n such that xS = 0, x[n]\S = 0 and F ⊤ x = 0, then for any (x, z, t) ∈ X we find that lim…”

“…Note that condition W W † x = x is used to enforce the complementarity constraints. We point out that a similar idea was recently used in the context of low-rank optimization [10]. Now consider the convex relaxation of (3), obtained by dropping the integrality constraints z ∈ {0, 1} n :…”

“…Recall that π S : R n → R S is the projection onto the subspace indexed by S. Now consider the natural convex relaxation of (10),…”

A new perspective on low-rank optimization

“…Existing attempts at solving this problem generally involve replacing the low-rank term with a nuclear norm term [44], which succeeds under some strong assumptions on the problem data but not in general. Recently, [8] proposed a new framework to model rank constraints, using orthogonal projection matrices which satisfy Y 2 = Y instead of binary variables which satisfy z 2 = z. By building on their work, in this paper we propose a generalization of the perspective function to matrix-valued functions and develop a matrix analog of the perspective reformulation technique from MIO which uses projection matrices instead of binary variables.…”

“…Combined with our MPRT, it leads to the relaxation: Of the three classes of approaches, heuristics currently dominate the literature, because their superior runtime and memory usage allows them to address larger-scale problems. However, recent advances in algorithmic theory and computational power have drastically improved the scalability of exact and approximate methods, to the point where they can now solve moderately sized problems which are relevant in practice [8]. Moreover, relaxations of strong exact formulations often give rise to very efficient heuristics (via tight relaxations of the exact formulation) which outperform existing heuristics.…”