The inherent feature of the Least Mean Squares (LMS) algorithm is the step size, and it requires careful adjustment. Small step size, required for small excess mean square error, results in slow convergence. Large step size, needed for fast adaptation, may result in loss of stability. Therefore, many modifications of the LMS algorithm, where the step size changes during the adaptation process depending on some particular characteristics, were and are still being developed.The paper reviews seventeen of the best known variable step-size LMS (VS-LMS) algorithms to the degree of detail that allows to implement them. The performance of the algorithms is compared in three typical applications: parametric identification, line enhancement, and adaptive noise cancellation. The paper suggests also one general modification that can simplify the choice of the upper bound for the step size, which is a crucial parameter for many VS-LMS algorithms. u(n)y
In the areas of acoustic research or applications that deal with not-precisely-known or variable conditions, a method of adaptation to the uncertainness or changes is usually necessary. When searching for an adaptation algorithm, it is hard to overlook the least mean squares (LMS) algorithm. Its simplicity, speed of computation, and robustness has won it a wide area of applications: from telecommunication, through acoustics and vibration, to seismology. The algorithm, however, still lacks a full theoretical analysis. This is probabely the cause of its main drawback: the need of a careful choice of the step size -which is the reason why so many variable step size flavors of the LMS algorithm has been developed. This paper contributes to both the above mentioned characteristics of the LMS algorithm. First, it shows a derivation of a new necessary condition for the LMS algorithm convergence. The condition, although weak, proved useful in developing a new variable step size LMS algorithm which appeared to be quite different from the algorithms known from the literature. Moreover, the algorithm proved to be effective in both simulations and laboratory experiments, covering two possible applications: adaptive line enhancement and active noise control.
SummaryA sufficient condition for least mean squares (LMS) algorithm stability with a small set of assumptions is derived in this paper. The derivation is not, contrary to the majority of currently known conditions, based on the independence assumption or other statistic properties of the input signals. Moreover, it does not make use of the small‐step‐size assumption, neither does it assume the input signals are stationary. Instead, it uses a theory of discrete systems and properties of a discrete state‐space matrix. Therefore, the result can be applied to a wide set of signals, including deterministic and nonstationary signals. The location of all eigenvalues of the matrix responsible for the LMS algorithm stability has been calculated. Simulation experiments, where the step size reaches a couple of hundreds without loss of stability, are shown to support the theory. On the other hand, simulation where the calculations based on the small‐step‐size theory provide a too large estimation of the upper bound for the step size, while the new condition gives a proper solution, is also presented. Therefore, the new condition may be used in cases where fast adaptation is necessary and when the independence theory or the small‐step‐size assumptions do not hold. Copyright © 2014 John Wiley & Sons, Ltd.
The Least Mean Square (LMS) algorithm and its variants are currently the most frequently used adaptation algorithms; therefore, it is desirable to understand them thoroughly from both theoretical and practical points of view. One of the main aspects studied in the literature is the influence of the step size on stability or convergence of LMS-based algorithms. Different publications provide different stability upper bounds, but a lower bound is always set to zero. However, they are mostly based on statistical analysis. In this paper we show, by means of control theoretic analysis confirmed by simulations, that for the leaky LMS algorithm, a small negative step size is allowed. Moreover, the control theoretic approach alows to minimize the number of assumptions necessary to prove the new condition. Thus, although a positive step size is fully justified for practical applications since it reduces the mean-square error, knowledge about an allowed small negative step size is important from a cognitive point of view.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.