Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal

A new reweighted l 1 -norm penalized least mean square (LMS) algorithm for sparse channel estimation is proposed and studied in this paper. Since standard LMS algorithm does not take into account the sparsity information about the channel impulse response (CIR), sparsity-aware modifications of the LMS algorithm aim at outperforming the standard LMS by introducing a penalty term to the standard LMS cost function which forces the solution to be sparse. Our reweighted l 1 -norm penalized LMS algorithm introduces in addition a reweighting of the CIR coefficient estimates to promote a sparse solution even more and approximate l 0 -pseudo-norm closer. We provide in depth quantitative analysis of the reweighted l 1 -norm penalized LMS algorithm. An expression for the excess mean square error (MSE) of the algorithm is also derived which suggests that under the right conditions, the reweighted l 1 -norm penalized LMS algorithm outperforms the standard LMS, which is expected. However, our quantitative analysis also answers the question of what is the maximum sparsity level in the channel for which the reweighted l 1 -norm penalized LMS algorithm is better than the standard LMS. Simulation results showing the better performance of the reweighted l 1 -norm penalized LMS algorithm compared to other existing LMS-type algorithms are given.

show abstract

“…Several sparsity-aware modifications of the standard LMS have been introduced in the literature [7,8,9,10,11,12,13,24].…”

Section: Standard Lmsmentioning

confidence: 99%

“…A penalty in the form of the l 0 -pseudo-norm of the CIR is used in [8], while [7] uses the l 1 -norm. In [9], the mean square convergence and stability analysis for one of the algorithms in [7] for the case of white input signals is presented. A performance analysis of the l 0 -pseudo-norm constraint LMS algorithm of [8] is given in [10].…”

Section: Introductionmentioning

confidence: 99%

Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis

Taheri

Vorobyov

2014

Signal Processing

View full text Add to dashboard Cite

show abstract

“…The first sparse LMS algorithm motivated by the CS technique is carried out by introducing a l 1 -norm constraint term into the basic LMS to exploit the in-nature sparse characteristics of broadband multi-path channel [24][25][26][27]. As a result, a zero attractor is given in the updating equation of the sparse LMS algorithm to put forward a zero-attracting (ZA) LMS (ZA-LMS) algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…As a result, a zero attractor is given in the updating equation of the sparse LMS algorithm to put forward a zero-attracting (ZA) LMS (ZA-LMS) algorithm. Furthermore, a reweighting ZA-LMS (RZA-LMS) was reported by using a sum-log constraint instead of the l 1 -norm penalty in the ZA-LMS algorithm [24,27]. Subsequently, the zero attracting techniques have been widely researched, and a great quantity of sparse LMS algorithms was exploited by using different norm constraints, such as l p -norm and smooth approximation l 0 -norm constraints [28][29][30][31][32].…”

Section: Introductionmentioning

confidence: 99%

“…Their estimation behaviors are discussed via a broadband multi-path channel, which is a prior known sparse channel from the previous studies [6,7,9,12,14,24,[27][28][29][30][31][32][33][34][35][36][37][38][39][40]. The proposed algorithms are realized by implementing a joint-optimization on the regularization parameter and step size to achieve a minimization of the channel estimation misalignment, where the joint-optimization method is obtained from [41].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Norm Penalized Joint-Optimization NLMS Algorithms for Broadband Sparse Adaptive Channel Estimation

Wang

2017

Symmetry

View full text Add to dashboard Cite

Abstract:A joint-optimization method is proposed for enhancing the behavior of the l 1 -norm-and sum-log norm-penalized NLMS algorithms to meet the requirements of sparse adaptive channel estimations. The improved channel estimation algorithms are realized by using a state stable model to implement a joint-optimization problem to give a proper trade-off between the convergence and the channel estimation behavior. The joint-optimization problem is to optimize the step size and regularization parameters for minimizing the estimation bias of the channel. Numerical results achieved from a broadband sparse channel estimation are given to indicate the good behavior of the developed joint-optimized NLMS algorithms by comparison with the previously proposed l 1 -normand sum-log norm-penalized NLMS and least mean square (LMS) algorithms.

show abstract

Extension of LMS stability condition over a wide set of signals

Bismor

2014

Adaptive Control & Signal

View full text Add to dashboard Cite

SummaryA sufficient condition for least mean squares (LMS) algorithm stability with a small set of assumptions is derived in this paper. The derivation is not, contrary to the majority of currently known conditions, based on the independence assumption or other statistic properties of the input signals. Moreover, it does not make use of the small‐step‐size assumption, neither does it assume the input signals are stationary. Instead, it uses a theory of discrete systems and properties of a discrete state‐space matrix. Therefore, the result can be applied to a wide set of signals, including deterministic and nonstationary signals. The location of all eigenvalues of the matrix responsible for the LMS algorithm stability has been calculated. Simulation experiments, where the step size reaches a couple of hundreds without loss of stability, are shown to support the theory. On the other hand, simulation where the calculations based on the small‐step‐size theory provide a too large estimation of the upper bound for the step size, while the new condition gives a proper solution, is also presented. Therefore, the new condition may be used in cases where fast adaptation is necessary and when the independence theory or the small‐step‐size assumptions do not hold. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract

Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal

Cited by 139 publications

References 6 publications

Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis

Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis

Norm Penalized Joint-Optimization NLMS Algorithms for Broadband Sparse Adaptive Channel Estimation

Extension of LMS stability condition over a wide set of signals

Contact Info

Product

Resources

About