Why Gabor frames? Two fundamental measures of coherence and their role in model selection

Abstract

The problem of model selection arises in a number of contexts, such as subset selection in linear regression, estimation of structures in graphical models, and signal denoising. This paper studies non-asymptotic model selection for the general case of arbitrary (random or deterministic) design matrices and arbitrary nonzero entries of the signal.In this regard, it generalizes the notion of incoherence in the existing literature on model selection and introduces two fundamental measures of coherence-termed as the worst-case coherence and the average coherence-among the columns of a design matrix. It utilizes these two measures of coherence to provide an in-depth analysis of a simple, model-order agnostic one-step thresholding (OST) algorithm for model selection and proves that OST is feasible for exact as well as partial model selection as long as the design matrix obeys an easily verifiable property, which is termed as the coherence property. One of the key insights offered by the ensuing analysis in this regard is that OST can successfully carry out model selection even when methods based on convex optimization such as the lasso fail due to the rank deficiency of the submatrices of the design matrix. In addition, the paper establishes that if the design matrix has reasonably small worst-case and average coherence then OST performs near-optimally when either (i) the energy of any nonzero entry of the signal is close to the average signal energy per nonzero entry or (ii) the signal-to-noise ratio in the measurement system is not too high. Finally, two other key contributions of the paper are that (i) it provides bounds on the average coherence of Gaussian matrices and Gabor frames, and (ii) it extends the results on model selection using OST to low-complexity, model-order agnostic recovery of sparse signals with arbitrary nonzero entries. In particular, this part of the analysis in the paper implies that an Alltop Gabor frame together with OST can successfully carry out model selection and recovery of sparse signals irrespective of the phases of the nonzero entries even if the number of nonzero entries scales almost linearly with the number of rows of the Alltop Gabor frame. Input: An n × p matrix X, a vector y ∈ C n , and a threshold λ > 0 Output: An estimate S ⊂ {1, . . . , p} of the true model S f ← X H y {Form signal proxy} S ← {i ∈ {1, . . . , p} : |f i | > λ} {Select model via OST} Roughly speaking, worst-case coherence-which seems to have been introduced in the related literature in [4]-isa similarity measure between the columns of a design matrix: the smaller the worst-case coherence, the less similar the columns. On the other hand, average coherence-which was first introduced in a conference version of this paper [5]-is a measure of the spread of the columns of a design matrix within the n-dimensional unit ball: the smaller the average coherence, the more spread out the column vectors.Our main contribution in the area of model selection is that we make use of these two measures of coherence to propose and...

Robust support recovery using sparse compressive sensing matrices

“…In addition to the connection with existing work in support recovery mentioned above [9]- [15], [23], the results established here also compliment a growing body of work in sparse recovery using sparse measurement matrices. A comprehensive overview of recent work in this field is provided in [30]; here we provide a brief (and necessarily incomplete) list of a few related works.…”

“…A common choice in the literature is to model the uncertainty by zero-mean additive white Gaussian noise, giving rise to an observation model of the form y = Ax + w, where w ∼ N (0, σ 2 Im×m); several existing works have examined support recovery procedures in such settings [9]- [15].…”

Reduced-dimension multiuser detection

“…The proof for the RDD detector is obtained by combining Lemmas 2, 3 and 4. Lemma 2 ensures that the event G occurs with probability at least as high as one minus (30). Whenever G occurs, Lemma 3 guarantees by using (13), that the RDD detector can correctly detect active users under the condition (29), i.e.…”

Gabor fusion frames generated by difference sets

“…Gabor frames [21], which are collections of time-and frequency-shifts (translations and modulations) of a chosen generator, have been shown useful for a variety of applications in signal processing related to signals sparse in a Gabor system, for example, model selection (also called sparsity pattern recovery) [4], and channel estimation and identification [22]. A crucial property of a Gabor system is that for any unitnorm nonzero generator v ∈ C N , it constitutes an N -tight frame [20,21].…”