An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem

Abstract

Given a large overcomplete dictionary of basis vectors, the goal is to simultaneously represent 1 signal vectors using coefficient expansions marked by a common sparsity profile. This generalizes the standard sparse representation problem to the case where multiple responses exist that were putatively generated by the same small subset of features. Ideally, the associated sparse generating weights should be recovered, which can have physical significance in many applications (e.g., source localization). The generic solution to this problem is intractable and, therefore, approximate procedures are sought. Based on the concept of automatic relevance determination, this paper uses an empirical Bayesian prior to estimate a convenient posterior distribution over candidate basis vectors. This particular approximation enforces a common sparsity profile and consistently places its prominent posterior mass on the appropriate region of weight-space necessary for simultaneous sparse recovery. The resultant algorithm is then compared with multiple response extensions of matching pursuit, basis pursuit, FOCUSS, and Jeffreys prior-based Bayesian methods, finding that it often outperforms the others. Additional motivation for this particular choice of cost function is also provided, including the analysis of global and local minima and a variational derivation that highlights the similarities and differences between the proposed algorithm and previous approaches.

Spatio-temporal reconstruction of brain dynamics from EEG with a Markov prior

“…T-MSBL (Zhang and Rao, 2011) is an extension of the multiple measurement vectors (MMV) sparse Bayesian learning (SBL) (Wipf and Rao, 2007) method that exploits temporal correlation to obtain smooth temporal dynamics. T-MSBL assumes 180 a block-structure where temporal correlations are modeled in the blocks.…”

“…Although no gold standard EEG inverse solver has been established, the field is con-20 verging on methods that employ spatial sparsity (Gorodnitsky and Rao, 1997;Wipf and Rao, 2007;Vega-Hernández et al, 2008;Friston et al, 2008;Zhang and Rao, 2011;Stahlhut et al, 2011;Montoya-Martinez et al, 2012;Gramfort et al, 2013;Hansen et al, 2013c;Hansen and Hansen, 2014;Andersen et al, 2014). Evidence was presented, in recent work (Delorme et al, 2012) that the instantaneous independent components of EEG 25 signals are dipolar and localized.…”

“…To stabilize the solution it is useful to impose some level of temporal smoothness. A basic scheme is to enforce that the locations of activity are 50 fixed throughout an analysis window (Wipf and Rao, 2007;Friston et al, 2008;Ou et al, 2009;Zhang and Rao, 2011;Hansen et al, 2013c). While useful for short time windows, this may be less appropriate for more extended and non-stationary settings.…”

“…ARD is in SBL used to prune away variables by assigning a hyperparameter to each variable dictating whether to keep or discard the variable. Extending SBL to the MMV framework one hyperparameter controls all the time samples for each variable (Wipf and Rao, 2007).…”

“…As more samples are available in determining whether a variable is relevant in the MMV 330 model an improved solution is obtained (Wipf and Rao, 2007;Zhang and Rao, 2011).…”