Orthogonal least squares learning algorithm for radial basis function networks

Abstract

The radial basis function network offers a viable alternative to the two-layer neural network in many applications of signal processing. A common learning algorithm for radial basis function networks is based on first choosing randomly some data points as radial basis function centers and then using singular-value decomposition to solve for the weights of the network. Such a procedure has several drawbacks, and, in particular, an arbitrary selection of centers is clearly unsatisfactory. The authors propose an alternative learning procedure based on the orthogonal least-squares method. The procedure chooses radial basis function centers one by one in a rational way until an adequate network has been constructed. In the algorithm, each selected center maximizes the increment to the explained variance or energy of the desired output and does not suffer numerical ill-conditioning problems. The orthogonal least-squares learning strategy provides a simple and efficient means for fitting radial basis function networks. This is illustrated using examples taken from two different signal processing applications.

Comparing error minimized extreme learning machines and support vector sequential feed-forward neural networks

“…Very similar methods that also construct SLFNs sequentially had been reported earlier ( [3], [16], [12]). They all find the optimal linear weights of the output layer by solving the same linear system.…”

“…This method, which has been referred to as SV-SFNNs [13] or SAOCIF with Input strategy [12], is essentially equivalent to the Orthogonal Least Squares Learning algorithm [3] and to the Kernel Matching Pursuit with pre-fitting method [16]. In order to assess the relative performance of both approaches (EM-ELMs vs. SV-SFNNs) in a fair manner, an empirical study has been realized on twenty benchmark data sets, 10 for classification and 10 for regression, under the same conditions and using the same software.…”

“…b = 1 for RBF units and b = 0 for additive units). Then, if we set C max = L − N + 1 and generate the candidates using the input strategy, then the resulting method, which has been called SV-SFNNs [13], is equivalent to the Orthogonal Least Squares Learning (OLSL) algorithm [3] and to Kernel Matching Pursuit with pre-fitting (KMP-prefit) [16]. Actually, OLSL was only proposed for RBF units and KMP-prefit for kernel-based activation functions, while SAOCIF with input strategy permits as well any other activation function with universal approximation capabilities (e.g.…”

“…In this case, the resulting method is equivalent to the Orthogonal Least Squares Learning algorithm [3] and to Kernel Matching Pursuit with pre-fitting [16]. All of them select the hiddenlayer weights among the input vectors.…”

Neural network based detection of hard exudates in retinal images

“…However, their training involves learning not only the weights, but also the number of radial basis functions, the position of their centres and their width. To accomplish this task, we applied the orthogonal least squares algorithm [34]. The optimum spread and number of radial basis functions were experimentally determined.…”