Group-Sparse Model Selection: Hardness and Relaxations

Baldassarre, Luca; Bhan, Nirav; Cevher, Volkan; Kyrillidis, Anastasios; Satpathi, Siddhartha

doi:10.48550/arxiv.1303.3207

Cited by 7 publications

(27 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, observe that under the unit 2 restriction on the feasible vectors, the maximizations in ( 5) and ( 6) are equivalent to computing the Euclidean projection of a given real vector on the (nonconvex) sets U and V. Such exact or approximate projection procedures exist for several interesting constraints beyond sparsity such as smooth or group sparsity (Huang et al, 2011;Baldassarre et al, 2013;…”

Section: Beyond Sparsity: Structured Ccamentioning

confidence: 99%

A simple and provable algorithm for sparse diagonal CCA

Asteris¹,

Kyrillidis²,

Koyejo³

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

Given two sets of variables, derived from a common set of samples, sparse Canonical Correlation Analysis (CCA) seeks linear combinations of a small number of variables in each set, such that the induced canonical variables are maximally correlated. Sparse CCA is NP-hard.We propose a novel combinatorial algorithm for sparse diagonal CCA, i.e., sparse CCA under the additional assumption that variables within each set are standardized and uncorrelated. Our algorithm operates on a low rank approximation of the input data and its computational complexity scales linearly with the number of input variables. It is simple to implement, and parallelizable. In contrast to most existing approaches, our algorithm administers precise control on the sparsity of the extracted canonical vectors, and comes with theoretical data-dependent global approximation guarantees, that hinge on the spectrum of the input data. Finally, it can be straightforwardly adapted to other constrained variants of CCA enforcing structure beyond sparsity.We empirically evaluate the proposed scheme and apply it on a real neuroimaging dataset to investigate associations between brain activity and behavior measurements. 1 We assume that the variables in X and Y are standardized, i.e., each column has zero mean and has been scaled to have unit standard deviation.

show abstract

Section: Beyond Sparsity: Structured Ccamentioning

confidence: 99%

A simple and provable algorithm for sparse diagonal CCA

Asteris¹,

Kyrillidis²,

Koyejo³

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…An important group structure is given by loopless pairwise overlapping groups because it leads to tractable projections [5]. This group structure consists of groups such that each element of the ground set occurs in at most two groups and the induced graph does not contain loops.…”

Section: Sparse Group Modelsmentioning

confidence: 99%

“…I.e., if we know the group support of the solution, the entries' values are naturally given by the anchor point x. The authors in [5] prove that the group support can be obtained by solving a discrete problem, according to the next lemma.…”

Section: The Discrete Modelmentioning

confidence: 99%

See 1 more Smart Citation

Structured Sparsity: Discrete and Convex Approaches

Kyrillidis

Baldassarre

Halabi

et al. 2015

Compressed Sensing and Its Applications

Self Cite

View full text Add to dashboard Cite

Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the relevant information therein typically resides in a much lower dimensional space. However, many solutions proposed nowadays do not leverage the true underlying structure. Recent results in CS extend the simple sparsity idea to more sophisticated structured sparsity models, which describe the interdependency between the nonzero components of a signal, allowing to increase the interpretability of the results and lead to better recovery performance. In order to better understand the impact of structured sparsity, in this chapter we analyze the connections between the discrete models and their convex relaxations, highlighting their relative advantages. We start with the general group sparse model and then elaborate on two important special cases: the dispersive and the hierarchical models. For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations. We also consider more general structures as defined by set functions and present their convex proxies. Further, we discuss efficient optimization solutions for structured sparsity problems and illustrate structured sparsity in action via three applications.

show abstract

“…Model-based sparse recovery: Nevertheless, sparsity is merely a first-order description of β and in many applications we have considerably more information a priori. In this work, we consider the k-sparse block model [29,3,22]:…”

Section: Introductionmentioning

confidence: 99%

Convex block-sparse linear regression with expanders -- provably

Kyrillidis,

Bah,

Hasheminezhad

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

Sparse matrices are favorable objects in machine learning and optimization. When such matrices are used, in place of dense ones, the overall complexity requirements in optimization can be significantly reduced in practice, both in terms of space and run-time. Prompted by this observation, we study a convex optimization scheme for block-sparse recovery from linear measurements. To obtain linear sketches, we use expander matrices, i.e., sparse matrices containing only few non-zeros per column. Hitherto, to the best of our knowledge, such algorithmic solutions have been only studied from a non-convex perspective. Our aim here is to theoretically characterize the performance of convex approaches under such setting.Our key novelty is the expression of the recovery error in terms of the model-based norm, while assuring that solution lives in the model. To achieve this, we show that sparse model-based matrices satisfy a group version of the null-space property. Our experimental findings on synthetic and real applications support our claims for faster recovery in the convex setting -as opposed to using dense sensing matrices, while showing a competitive recovery performance.

show abstract

Group-Sparse Model Selection: Hardness and Relaxations

Cited by 7 publications

References 0 publications

A simple and provable algorithm for sparse diagonal CCA

A simple and provable algorithm for sparse diagonal CCA

Structured Sparsity: Discrete and Convex Approaches

Convex block-sparse linear regression with expanders -- provably

Contact Info

Product

Resources

About