Stochastic analysis of gradient adaptive identification of nonlinear systems with memory for Gaussian data and noisy input and output measurements

Abstract

This paper analyzes the statistical behavior of a sequential gradient search adaptive algorithm for identifying an unknown nonlinear system comprised of a discrete-time linear system H followed by a zero-memory nonlinearity g(.). The LMS algorithm tirst estimates H. The weights are then frozen. Recursions are derived for the mean and fluctuation behavior of LMS which agree with Monte Carlo simulations. When the nonlinearity is modelled by a scaled error function, the second part of the gradient scheme is shown to correctly learn the scale factor and the error function scale factor. Mean recursions for the scale factors show good agreement with Monte Carlo simulations.

Stochastic analysis of neural network modeling and identification of nonlinear memoryless MIMO systems

“…The calculations are similar to [9] Appendix, the main difference is that here we deal with a multi-dimensional input. Therefore, we will follow the same methodology as in [9].…”

“…Several authors have analyzed NN algorithms during the last two decades which considerably helped the neural network community to better understand the mechanisms of neural networks [1,[7][8][9][10][11][12][13][14][15]. For example, the authors in [13] have studied a simple structure consisting of two inputs and a single neuron.…”

“…The authors in [8] studied a memoryless single-input single-output (SISO) system identification model for the single neuron case. In [9] the authors proposed a stochastic analysis of gradient adaptive identification of nonlinear Wiener systems composed of a linear filter followed with a Zeromemory nonlinearity. The model was composed of a linear adaptive filter followed by an adaptive parameterized version of the nonlinearity.…”

Nonnegative Least-Mean-Square Algorithm

“…We have (24) Taking the expectation, defining , and neglecting the statistical dependence of and , we obtain (25) This implies that where denotes the vector whose th entry is defined by . Using these results with (22) yields the following mean weight-error vector update equation: (26) This equation requires second-order moments defined by the matrix in order to update the first-order one provided by .…”

“…(7)- (9)], using results from [25]. Following the same procedure as in [24] yields (63) Now, taking the expected value with respect to (64)…”