Evolving space-filling curves to distribute radial basis functions over an input space

Whitehead, Bruce A.; Choate, T.D.

doi:10.1109/72.265957

Cited by 97 publications

(32 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A very complete state of the art of the different approaches and characteristics of a wide range of EA and RBFNN combinations is given in [15]. For example, RBFNN design has been approached by using evolutionary clustering techniques with local search [16], hybrid algorithms [17], [18], or multiobjective algorithms [19], or by evolving only the basis functions [20], [21] or space-filling curves to generate the RBF centers [22].…”

Section: Related Workmentioning

confidence: 99%

Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks

Gutiérrez

Hervás-Martı́nez

Martínez-Estudillo

2011

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Abstract-This paper proposes a hybrid multilogistic methodology, named logistic regression using initial and radial basis function (RBF) covariates. The process for obtaining the coefficients is carried out in three steps. First, an evolutionary programming (EP) algorithm is applied, in order to produce an RBF neural network (RBFNN) with a reduced number of RBF transformations and the simplest structure possible. Then, the initial attribute space (or, as commonly known as in logistic regression literature, the covariate space) is transformed by adding the nonlinear transformations of the input variables given by the RBFs of the best individual in the final generation. Finally, a maximum likelihood optimization method determines the coefficients associated with a multilogistic regression model built in this augmented covariate space. In this final step, two different multilogistic regression algorithms are applied: one considers all initial and RBF covariates (multilogistic initial-RBF regression) and the other one incrementally constructs the model and applies cross validation, resulting in an automatic covariate selection [simplelogistic initial-RBF regression (SLIRBF)]. Both methods include a regularization parameter, which has been also optimized. The methodology proposed is tested using 18 benchmark classification problems from well-known machine learning problems and two real agronomical problems. The results are compared with the corresponding multilogistic regression methods applied to the initial covariate space, to the RBFNNs obtained by the EP algorithm, and to other probabilistic classifiers, including different RBFNN design methods [e.g., relaxed variable kernel density estimation, support vector machines, a sparse classifier (sparse multinomial logistic regression)] and a procedure similar to SLIRBF but using product unit basis functions. The SLIRBF models are found to be competitive when compared with the corresponding multilogistic regression methods and the RBFEP method. A measure of statistical significance is used, which indicates that SLIRBF reaches the state of the art.

show abstract

Section: Related Workmentioning

confidence: 99%

Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks

Gutiérrez

Hervás-Martı́nez

Martínez-Estudillo

2011

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

show abstract

“…Alternatively, all the parameters of the RBF network, the RBF centres, variances or covariance matrices, and weights, can be learnt together via a non-linear optimisation [24] . The optimisation process associated with this nonlinear learning approach, however, is highly complex and non-convex, and the genetic algorithm (GA) has been suggested to solve this type of nonlinear learning problems [25] , at the cost of an increased computational complexity.…”

Section: Introductionmentioning

confidence: 99%

Using the correlation criterion to position and shape RBF units for incremental modelling

Wang

Chen

Harris

2006

Int J Automat Comput

View full text Add to dashboard Cite

A novel technique is proposed for the incremental construction of sparse radial basis function (RBF) networks. The correlation between a RBF regressor and the training data is used as the criterion to position and shape the RBF node, and it is shown that this is equivalent to incrementally minimise the modelling mean square error. A guided random search optimisation method, called the repeated weighted boosting search, is adopted to append RBF nodes one by one in an incremental regression modelling procedure. The experimental results obtained using the proposed method demonstrate that it provides a viable alternative to the existing state-of-the-art modelling techniques for constructing parsimonious RBF models that generalise well.

show abstract

“…The parameters of the RBF network, which include the center vectors and variances or covariance matrices of its hidden nodes, as well as the weights that connect the RBF nodes to the network output, can be trained together via nonlinear optimization using gradient based algorithms [27]- [31], the expectation-maximization (EM) algorithm [32], [33], or various evolutionary algorithms [34]- [38]. Generally speaking, learning based on such a nonlinear approach is computationally expensive and may encounter the problem of local minima.…”

mentioning

confidence: 99%

Particle Swarm Optimization Aided Orthogonal Forward Regression for Unified Data Modeling

Chen

Hong

Harris

2010

IEEE Trans. Evol. Computat.

View full text Add to dashboard Cite

Abstract-We propose a unified data modeling approach that is equally applicable to supervised regression and classification applications, as well as to unsupervised probability density function estimation. A particle swarm optimization (PSO) aided orthogonal forward regression (OFR) algorithm based on leaveone-out (LOO) criteria is developed to construct parsimonious radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines the center vector and diagonal covariance matrix of one RBF node by minimizing the LOO statistics. For regression applications, the LOO criterion is chosen to be the LOO mean square error, while the LOO misclassification rate is adopted in two-class classification applications. By adopting the Parzen window estimate as the desired response, the unsupervised density estimation problem is transformed into a constrained regression problem. This PSO aided OFR algorithm for tunable-node RBF networks is capable of constructing very parsimonious RBF models that generalize well, and our analysis and experimental results demonstrate that the algorithm is computationally even simpler than the efficient regularization assisted orthogonal least square algorithm based on LOO criteria for selecting fixed-node RBF models. Another significant advantage of the proposed learning procedure is that it does not have learning hyperparameters that have to be tuned using costly cross validation. The effectiveness of the proposed PSO aided OFR construction procedure is illustrated using several examples taken from regression and classification, as well as density estimation applications.

show abstract

Evolving space-filling curves to distribute radial basis functions over an input space

Cited by 97 publications

References 24 publications

Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks

Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks

Using the correlation criterion to position and shape RBF units for incremental modelling

Particle Swarm Optimization Aided Orthogonal Forward Regression for Unified Data Modeling

Contact Info

Product

Resources

About