An Efficient Sequential Learning Algorithm for Growing and Pruning RBF (GAP-RBF) Networks

Abstract

This paper presents a simple sequential growing and pruning algorithm for radial basis function (RBF) networks. The algorithm referred to as growing and pruning (GAP)-RBF uses the concept of "Significance" of a neuron and links it to the learning accuracy. "Significance" of a neuron is defined as its contribution to the network output averaged over all the input data received so far. Using a piecewise-linear approximation for the Gaussian function, a simple and efficient way of computing this significance has been derived for uniformly distributed input data. In the GAP-RBF algorithm, the growing and pruning are based on the significance of the "nearest" neuron. In this paper, the performance of the GAP-RBF learning algorithm is compared with other well-known sequential learning algorithms like RAN, RANEKF, and MRAN on an artificial problem with uniform input distribution and three real-world nonuniform, higher dimensional benchmark problems. The results indicate that the GAP-RBF algorithm can provide comparable generalization performance with a considerably reduced network size and training time.

GAP-RBF Based NR Image Quality Measurement for JPEG Coded Images

“…In recent years, many sequential learning algorithms have been developed to overcome the problems encountered in neural networks [10,11,12]. Here, radial basis function network (RBF) is used to approximate the functional relationship.…”

“…Recently Huang et al, [12] proposed a new sequential learning algorithm called 'growing and pruning radial-basis function (GAP-RBF) network. In this algorithm, the criteria for growing/pruning of hidden neurons is based on the significance of the neurons to the network output.…”

“…The learning algorithm uses 'growing and pruning' strategy to decide on the significance of a neuron towards realizing a compact model. The network parameters such as center vectors, connection weights and widths of hidden neurons are tuned using extended Kalman filter (EKF) algorithm [12].…”

“…The algorithm adds/prunes hidden neurons and also tunes the network parameters. Detailed description of the the algorithms can be found in [12]. The following are the steps involved in obtaining a compact network.…”

“…ELSE Update the network parameters α nr , μ nr and σ nr for the nearest neuron only, using EKF algorithm [12]. Criterion for pruning the hidden neurons:…”