Linear Regression With a Sparse Parameter Vector

Larsson, Erik G.; Selén, Y.

doi:10.1109/tsp.2006.887109

Cited by 81 publications

(100 citation statements)

References 17 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…A Gaussian mixture model similar to that in Section II was adopted by Larsson and Selén [14], who, for Q = 2, also constructed the MMSE estimate in the manner of (7) but with an S that contains exactly one sequence s for each Hamming weight 0 to N . They proposed to find these s by starting with an all-active basis configuration and recursively deactivating one element at a time.…”

Section: Discussion: Related Workmentioning

confidence: 99%

“…They proposed to find these s by starting with an all-active basis configuration and recursively deactivating one element at a time. Thus, the D = 1 version of the FBMP algorithm recalls the heuristic of [14], but in reverse. The fast update presented here has a complexity of only O(N M P ), in comparison to O(N 3 M 2 ) for the technique in [14].…”

Section: Discussion: Related Workmentioning

confidence: 99%

“…Thus, the D = 1 version of the FBMP algorithm recalls the heuristic of [14], but in reverse. The fast update presented here has a complexity of only O(N M P ), in comparison to O(N 3 M 2 ) for the technique in [14]. Given the typically large values of N encountered in practice, the complexity of FBMP can be several orders of magnitude lower.…”

Section: Discussion: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A fast posterior update for sparse underdetermined linear models

Potter

Schniter

Ziniel

2008

2008 42nd Asilomar Conference on Signals, Systems and Computers

View full text Add to dashboard Cite

Abstract-A Bayesian approach is adopted for linear regression, and a fast algorithm is given for updating posterior probabilities. Emphasis is given to the underdetermined and sparse case, i.e., fewer observations than regression coefficients and the belief that only a few regression coefficients are non-zero. The fast update allows for a low-complexity method of reporting a set of models with high posterior probability and their exact posterior odds. As a byproduct, this Bayesian model averaged approach yields the minimum mean squared error estimate of unknown coefficients. Algorithm complexity is linear in the number of unknown coefficients, the number of observations and the number of nonzero coefficients. For the case in which hyperparameters are unknown, a maximum likelihood estimate is found by a generalized expectation maximization algorithm.

show abstract

Section: Discussion: Related Workmentioning

confidence: 99%

Section: Discussion: Related Workmentioning

confidence: 99%

Section: Discussion: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A fast posterior update for sparse underdetermined linear models

Potter

Schniter

Ziniel

2008

2008 42nd Asilomar Conference on Signals, Systems and Computers

View full text Add to dashboard Cite

show abstract

“…This model has also been used in [16] for simultaneously modeling the impulse and background noise in a real-field channel coding system. It is also used to model the impulse noise in a communication channel [45], or the sparse vector in an application of regression [46]. This model is also suitable for sparse decomposition applications where we want to decompose a signal as a combination of only a few atoms of the dictionary, while the coefficients of the other atoms are zero.…”

Section: System Modelmentioning

confidence: 99%

“…Hence, we must choose the value as . Consequently, to be sure that with a probability close to 1 the convergence is guaranteed, should be selected in the interval: (46) By imposing the condition of normalized column to , the diagonal elements of is equal to one and hence we have . Therefore, the suitable interval can be simplified as (…”

Section: B Convergence Of the Steepest-ascentmentioning

confidence: 99%

An Iterative Bayesian Algorithm for Sparse Component Analysis in Presence of Noise

Zayyani

Babaie‐Zadeh

Jutten

2009

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Abstract-We present a Bayesian approach for Sparse Component Analysis (SCA) in the noisy case. The algorithm is essentially a method for obtaining sufficiently sparse solutions of underdetermined systems of linear equations with additive Gaussian noise. In general, an underdetermined system of linear equations has infinitely many solutions. However, it has been shown that sufficiently sparse solutions can be uniquely identified. Our main objective is to find this unique solution. Our method is based on a novel estimation of source parameters and maximum a posteriori (MAP) estimation of sources. To tackle the great complexity of the MAP algorithm (when the number of sources and mixtures become large), we propose an Iterative Bayesian Algorithm (IBA). This IBA algorithm is based on the MAP estimation of sources, too, but optimized with a steepest-ascent method. The convergence analysis of the IBA algorithm and its convergence to true global maximum are also proved. Simulation results show that the performance achieved by the IBA algorithm is among the best, while its complexity is rather high in comparison to other algorithms. Simulation results also show the low sensitivity of the IBA algorithm to its simulation parameters.Index Terms-Atomic decomposition, blind source separation (BSS), sparse component analysis (SCA), sparse decomposition, sparse source separation.

show abstract

MAP versus MMSE Estimation

Elad

2010

Sparse and Redundant Representations

View full text Add to dashboard Cite

Linear Regression With a Sparse Parameter Vector

Cited by 81 publications

References 17 publications

A fast posterior update for sparse underdetermined linear models

A fast posterior update for sparse underdetermined linear models

An Iterative Bayesian Algorithm for Sparse Component Analysis in Presence of Noise

MAP versus MMSE Estimation

Contact Info

Product

Resources

About