We describe general heuristics to approximately solve a wide variety of problems with convex objective and decision variables from a nonconvex set. The heuristics, which employ convex relaxations, convex restrictions, local neighbor search methods, and the alternating direction method of multipliers (ADMM), require the solution of a modest number of convex problems, and are meant to apply to general problems, without much tuning. We describe an implementation of these methods in a package called NCVX, as an extension of CVXPY, a Python package for formulating and solving convex optimization problems. We study several examples of well known nonconvex problems, and show that our general purpose heuristics are effective in finding approximate solutions to a wide variety of problems.is often a Cartesian product, C = C 1 × · · · × C k , where C i ⊂ R q i are sets that are simple to describe, e.g., C i = {0, 1}. We denote the optimal value of the problem (1) as p , with the usual conventions that p = +∞ if the problem is infeasible, and p = −∞ if the problem is unbounded below.
Special casesMixed-integer convex optimization. When C = {0, 1} q , the problem (1) is a general mixed integer convex program, i.e., a convex optimization problem in which some variables are constrained to be Boolean. (Mixed Boolean convex program would be a more accurate name for such a problem, but 'mixed integer' is commonly used.) It follows that the problem (1) is hard; it includes as a special case, for example, the general Boolean satisfaction problem.Cardinality constrained convex optimization. As another broad special case of (1),is the number of nonzero elements of z, and k and M are given. We call this the general cardinality-constrained convex problem. It arises in many interesting applications, such as regressor selection.Other special cases. As we will see in §6, many (hard) problems can be formulated in the form (1). More examples include regressor selection, 3-SAT, circle packing, the traveling salesman problem, factor analysis modeling, job selection, the maximum coverage problem, inexact graph isomorphism, and many more.
Convex relaxationConvex relaxation of a set. A compact set C always has a tractable convex relaxation. By this we mean a (modest-sized) set of convex inequality and linear equality constraints that hold for every z ∈ C:We will assume that these relaxation constraints are included in the convex constraints of (1). Adding these relaxation constraints to the original problem yields an equivalent problem (since the added constraints are redundant), but can improve the convergence of any method, global or heuristic. By tractable, we mean that the number of added constraints is modest, and in particular, polynomial in q. For example, when C = {0, 1} q , we have the inequalities 0 ≤ z i ≤ 1, i = 1, . . . , q. (These inequalities define the convex hull of C, i.e., all other convex inequalities that hold for all z ∈ C are implied by them.) When