Memory-Efficient Structured Convex Optimization via Extreme Point Sampling

Abstract

Memory is a key computational bottleneck when solving large-scale convex optimization problems, such as semidefinite programs (SDPs). In this paper, we focus on the regime in which storing an n \times n matrix decision variable is prohibitive. To solve SDPs in this regime, we develop a randomized algorithm that returns a random vector whose covariance matrix is near-feasible and near-optimal for the SDP. We show how to develop such an algorithm by modifying the Frank--Wolfe algorithm to systematically replace the matrix iterates with random vectors. As an application of this approach, we show how to implement the Goemans--Williamson approximation algorithm for MaxCut using \scrO (n) memory in addition to the memory required to store the problem instance. We then extend our approach to deal with a broader range of structured convex optimization problems, replacing decision variables with random extreme points of the feasible region.

Accelerated first-order methods for a class of semidefinite programs

“…In our study, we will examine general SDPs satisfying similar structural assumptions and design an efficient storage-optimal FOM to solve them. In this respect, our approach also extends a recent line of work [23,25,53,68] towards developing storage-optimal FOMs for SDPs possessing low-rank solutions. We discuss storage optimality in SDP algorithms in Section 1.3.…”

“…Storage-optimal/efficient FOMs. A growing body of literature, itself containing multiple research strands, has explored FOMs for SDPs [5,9,20,23,25,34,37,38,47,53,56,[66][67][68]. Below, we recount some recent developments in this direction with a particular view towards storage-efficient or storage-optimal FOMs for SDPs admitting low-rank solutions.…”

“…In all, they show that it is possible to implement their algorithm in O −2 iterations where each iteration involves computing an eigenvector via Õ −1/2 matrix-vector products. In follow-up work, Shinde et al [53] combine the algorithmic architecture of SketchyCGAL with the additional observation that in specific applications (e.g., max-cut), the goal is simply to sample from a Gaussian distribution with variance given by an approximate solution Y to the SDP. Under this alternate goal, it is possible to further reduce the storage requirements to O(n + m).…”