Kernel Recursive Least Squares for the CMAC Neural Network

Abstract

Abstract-The Cerebellar Model Articulation Controller (CMAC) neural network is an associative memory that is biologically inspired by the cerebellum, which is found in the brains of animals. The standard CMAC uses the least mean squares algorithm to train the weights. Recently, the recursive least squares algorithm was proposed as a superior algorithm for training the CMAC online as it can converge in one epoch, and does not require tuning of a learning rate. However, the RLS algorithms computational speed is dependant on the number of weights required by the CMAC which is often large and thus can be very computationally inefficient. Recently also, the use of kernel methods in the CMAC was proposed to reduce memory usage and improve modeling capabilities. In this paper the Kernel Recursive Least Squares (KRLS) algorithm is applied to the CMAC. Due to the kernel method, the computational complexity of the CMAC becomes dependant on the number of unique training data, which can be significantly less than the weights required by non-kernel CMACs. Additionally, online sparsification techniques are applied to further improve computational speed.Index Terms-CMAC, kernel recursive least squares.

Cerebellar model articulation controller (CMAC) for sequential images coding

“…The gradient operator calculates direction to the minimum location of the performance function of � that, in this case, estimated as el, that is �=e { The convergence parameter is the step-size of adaptation to the minimum of the performance function of�. By applying (7) to (8), the gradient operator changes (9) to be as follows (10) After adaptation in the k th iteration, the network output can be expressed as follows (11 ) and the error is as below ( 12) The error is reduced if the convergence parameter Il is in the interval of (13)…”

“…In the study, the improvement occurred but the convergence aspects which related to the respective application constraints need to be investigated further. Kernel Recursive Least Squares Algorithm, which shown in [9], [10], [11], is used to improve the CMAC training through a utilization of the inverse QR decomposition [12] in the weight updating rule to increase the column independence of the basis function matrix, iteratively. The improvement is achieved but the need of computationally resources is still to be a problem.…”