Group-Sparse Model Selection: Hardness and Relaxations

Abstract

Abstract-Group-based sparsity models are instrumental in linear and non-linear regression problems. The main premise of these models is the recovery of "interpretable" signals through the identification of their constituent groups, which can also provably translate in substantial savings in the number of measurements for linear models in compressive sensing. In this paper, we establish a combinatorial framework for group-model selection problems and highlight the underlying tractability issues. In particular, we show that the group-model selection problem is equivalent to the well-known NP-hard weighted maximum coverage problem. Leveraging a graph-based understanding of group models, we describe group structures that enable correct model selection in polynomial time via dynamic programming. Furthermore, we show that popular group structures can be explained by linear inequalities involving totally unimodular matrices, which afford other polynomial time algorithms based on relaxations. We also present a generalization of the group model that allows for within group sparsity, which can be used to model hierarchical sparsity. Finally, we study the Pareto frontier between approximation error and sparsity budget of group-sparse approximations for two tractable models, among which the tree sparsity model, and illustrate selection and computation tradeoffs between our framework and the existing convex relaxations.

Uniform recovery of fusion frame structured sparse signals

“…This model is strongly related to (and can in fact be viewed as special case of) the joint sparsity model [21,22,26], where one considers a signal consisting of several "channels" (such as the three color channels of an RGB image) and assumes that nonzeros coefficients appear at the same location within each of the channels. A generalization of the block sparsity model is the group sparsity model where the groups of nonzero coefficients are allowed to overlap [3,34].…”

Learning-Based Compressive Subsampling

“…This does not correspond to a matroid constraint, but it can nevertheless be handled using dynamic programming [20].…”

Sparse group covers and greedy tree approximations

“…This problem is in general NP-hard, but there exist special cases, such as the block model [7] and the acyclic overlapping group model [5], which can be solved exactly in polynomial time. Furthermore, convex relaxations such as the group lasso [1], [2] and the latent group lasso [3], [4] provide tractable proxies to the discrete problem, with the disadvantage though of not being able to obtain the entire solution set [5].…”

“…This has led to the sparse group lasso norm [11] and also to the general dynamic program for acyclic group structures that finds a K-sparse solution covered by at most G-groups [5].…”

“…Two exact dynamic programs have been concurrently proposed recently for obtaining tree approximations with the same complexity [5], [6]. The hierarchical group lasso [8] norm enforces the same structure via a convex relaxation, with no direct control on the sparsity of the solution.…”