Bayesian methods for sparse RLS adaptive filters

Abstract

This work deals with an extension of the standard recursive least squares (RLS) algorithm. It allows to prune irrelevant coefficients of a linear adaptive filter with sparse impulse response and it provides a regularization method with automatic adjustment of the regularization parameter. New update equations for the inverse auto-correlation matrix estimate are derived that account for the continuing shrinkage of the matrix size. In case of densely populated impulse responses of length M , the computational complexity of the algorithm stays O(A1') as for standard RLS while for sparse impulse responses the new algorithm becomes much more efficient through the adaptive shrinkage of the dimension of the coefficient space. The algorithm has been successfully applied to the identification of sparse channel models (as in mobile radio or echo cancellation). INTRODUCTlONLinear-in-parameters models(1) with the observed noisy output z[nl, the weight vectorare considered. Many applications of these models share the features that the excitation signal Z[R] for the adaptive system i s not always persistently exciting and that the structure of the model does not match the structure of the reference system. One mismatch example would be a too high order of the adaptive filter. In the first case the covariance matrix estimate blows up such that the adaptive algorithm gets unstable. A common stabilization method for such situations is the regularization of the auto-correlation matrix estimate [l]. The second feature of model mismatch is due to the incomplete insight into the structure of the reference system. To guarantee some predefined error power after convergence, one has to select a conservative, i.e. overestimated. model structure which takes into account our G. PaoliSystem engineering group, Infineon Technologies, Design Center Villach, Austria gerhard.paoli@infineon.com incomplete knowledge about the reference system. In the case of an echo-canceler, where the echo-impulse response varies significantly over different environments, one has to initialize a conservative model which can handle the longest impulse-response expected to occur in practice. The inclusion of parameters in the model that are irrelevant from the viewpoint of a decrease in the error still causes an increase in the variance of the parameter estimates \t[n] of the adaptive system compared to the variance ofthe estimates for an exactly matching model structure. In addition, the tracking performance of the adaptive filter gets reduced due to the inclusion of irrelevant parameters.In the statistics and machine learning literature this problem gets addressed by subset selection algorithms. In this work the algorithm proposed in [2], which simultaneously performs subset selection and adaptive regularization, is incorporated in a recursive least squares adaptive algorithm.The Bayesian treatment of regularization using the evidence procedure [3] offers a simple way to estimate the regularization parameter and even allows an extension to estimate a re...

An efficient implementation of the Memory Improved Proportionate Affine Projection Algorithm

“…(28)- (31) imply an efficient way for the updating of M L ðnÞ from the previously computed matrix M L ðn À 1Þ, noting that (27) and the partition…”

“…This task corresponds to the inverse matrix down-dating (i.e., extraction of the inverse which corresponds to a lower order partition of a matrix) and it can be easily tackled using the matrix inversion lemma (see, e.g., [31][32][33]). Application of the matrix inversion lemma for partitioned matrices ((4.160) in [2]) in (33) results in…”

A Variational Bayes Framework for Sparse Adaptive Estimation

“…Notice also that the parameters of all variational distributions are expressed in terms of expectations of expressions of the other parameters. This gives rise to a variational iterative scheme, which involves updating (16), (17), for , (19) and (25), for , in a sequential manner. Due to the convexity of the factors , and , the variational Bayes algorithm converges to a sparse solution in a few cycles, [15].…”

“…Section II defines the mathematical formulation of the adaptive estimation 1 Note that a Bayesian approach to adaptive filtering has been previously proposed in [16]. However, in [16] a type-II maximum likelihood inference method is adopted that leads to a regularized RLS-type scheme. This is completely different from the approach and algorithms described in this work.…”

Identification of Nonlinear Oscillator Models for Speech Analysis and Synthesis

“…Other applications of the RVM algorithm comprise, for example, the determination of relevant weights in adaptive filters, where the complexity of the adaptive filters is also considerably reduced without impairing performance [38].…”