A Regularized Inverse QR Decomposition Based Recursive Least Squares Algorithm for the CMAC Neural Network

Abstract

Abstract-The Cerebellar Model Articulation Controller (CMAC) neural network is an associative memory that is biologically inspired by the cerebellum, which is found in the brains of animals. The standard CMAC uses the least mean squares algorithm (LMS) to train the weights. Recently, the recursive least squares (RLS) algorithm was proposed as a superior algorithm for training the CMAC online as it can converge in one epoch, and does not require tuning of a learning rate. However, the RLS algorithm was found to be very computationally demanding. In this work, the RLS computation time is reduced by using an inverse QR decomposition based RLS (IQR-RLS) algorithm which is also parallelized for multi-core CPUs. Furthermore, this work shows how the IQR-RLS algorithm may be regularized which greatly improves the generalization capabilities of the CMAC.

Kernel Recursive Least Squares for the CMAC Neural Network

“…Other RLS algorithm implementation variants such as a) b) c) QR-decomposition RLS (QRRLS) [9] and inverse QR-decomposition RLS (IQR-RLS) [4] have been used to improve the computational speed making CMAC-RLS feasible for low dimensional problems that require few weights.…”

“…Unfortunately, the number of weights required by the CMAC can be quite large for high dimensional problems. In [4] the inverse QR-RLS (IQRRLS) algorithm was used with the CMAC allowing real time RLS Manuscript received September 29, 2012; revised December 6, 2012. The authors are with the Department of Electrical and Electronic Engineering, University of Auckland, Auckland, New Zealand (e-mail: clau070@aucklanduni.ac.nz, g.coghill@auckland.ac.nz) learning of low dimensional problems (less than three dimensions) on a PC, although the algorithm is still too computationally demanding for the real time learning of higher dimensional problems.…”