Exact Sparse Approximation Problems via Mixed-Integer Programming: Formulations and Computational Performance

Abstract

Abstract-Sparse approximation addresses the problem of approximately fitting a linear model with a solution having as few non-zero components as possible. While most sparse estimation algorithms rely on suboptimal formulations, this work studies the performance of exact optimization of 0-norm-based problems through Mixed-Integer Programs (MIPs). Nine different sparse optimization problems are formulated based on 1, 2 or ∞ data misfit measures, and involving whether constrained or penalized formulations. For each problem, MIP reformulations allow exact optimization, with optimality proof, for moderate-size yet difficult sparse estimation problems. Algorithmic efficiency of all formulations is evaluated on sparse deconvolution problems. This study promotes error-constrained minimization of the 0 norm as the most efficient choice when associated with 1 and ∞ misfits, while the 2 misfit is more efficiently optimized with sparsity-constrained and sparsity-penalized problems. Then, exact 0-norm optimization is shown to outperform classical methods in terms of solution quality, both for over-and underdetermined problems. Finally, numerical simulations emphasize the relevance of the different p fitting possibilities as a function of the noise statistical distribution. Such exact approaches are shown to be an efficient alternative, in moderate dimension, to classical (suboptimal) sparse approximation algorithms with 2 data misfit. They also provide an algorithmic solution to less common sparse optimization problems based on 1 and ∞ misfits. For each formulation, simulated test problems are proposed where optima have been successfully computed. Data and optimal solutions are made available as potential benchmarks for evaluating other sparse approximation methods.

Feasibility Pump Algorithm for Sparse Representation under Laplacian Noise

“…This is an important improvement compared to the formulation (4) solved with the Branch and Cut algorithm. As reported in [17], Branch and Cut require between 3 to 10000 seconds to find the solution for the problem. So, SFPreg is significantly faster and still obtains results that are nearly optimal.…”

“…The big-trick is used as in [17], where is a preset parameter bounding the elements of , chosen as = 1.1‖ ‖ ∞ /‖ ‖ 2 2 . In this paper, the Feasibility Pump is adapted in order to obtain the sparse solution of problem (3).…”

“…We note that, unlike the Branch and Bound algorithms from [17], Algorithm 1 is not guaranteed to provide the exact solution of (1). However, it is significantly faster.…”

Nonconvex Regularization in Remote Sensing

“…The obvious choice is to use the 0 pseudonorm that returns directly the number of nonzero coefficients in w. Nevertheless, the 0 term is nonconvex and nondifferentiable and cannot be optimized exactly unless all of the possible subsets are tested. Despite recent works aiming at solving directly this problem via discrete optimization [51], this approach is still computationally impossible even for medium-sized problems. Greedy optimization methods have been proposed to solve this kind of optimization problem and have led to efficient algorithms such as orthogonal matching pursuit (OMP) [52] or orthogonal least square (OLS) [53].…”

Mixed Integer Programming For Sparse Coding: Application to Image Denoising

“…In this paper, following [15] and [16], the sparse coding in its exact 0 -norm formulation is recast as a mixed-integer quadratic programming (MIQP), namely a mixed-integer programming (MIP) with a quadratic objective function. MIP aims at solving optimization problems involving both integer and continuous variables.…”