GENERALIZED CELLULAR NEURAL NETWORKS (GCNNs) CONSTRUCTED USING PARTICLE SWARM OPTIMIZATION FOR SPATIO-TEMPORAL EVOLUTIONARY PATTERN IDENTIFICATION

Abstract

Particle swarm optimisation (PSO) is introduced to implement a new constructive learning algorithm for training generalised cellular neural networks (GCNNs) for the identification of spatiotemporal evolutionary (STE) systems. The basic idea of the new PSO-based learning algorithm is to successively approximate the desired signal by progressively pursuing relevant orthogonal projections.This new algorithm will thus be referred to as the orthogonal projection pursuit (OPP) algorithm, which is in mechanism similar to the conventional projection pursuit approach. A novel two-stage hybrid training scheme is proposed for constructing a parsimonious GCNN model. In the first stage, the orthogonal projection pursuit algorithm is applied to adaptively and successively augment the network, where adjustable parameters of the associated units are optimised using a particle swarm optimiser. The resultant network model produced at the first stage may be redundant. In the second stage, a forward orthogonal regression (FOR) algorithm, aided by mutual information estimation, is applied to refine and improve the initially trained network. The effectiveness and performance of the proposed method is validated by applying the new modelling framework to a spatio-temporal evolutionary system identification problem.

A robust model structure selection method for small sample size and multiple datasets problems

“…Note that different from traditional L2-norm based algorithms, e.g. the orthogonal projection pursuit (OPP) algorithm [28] that can be proven to converge, the proof of the convergence of the proposed RMSS method is not straightforward. In this study, the focus is on choosing a set of most powerful model terms from a given pool consisting of a large number of candidate model terms, through an iterative manner, one term at each search step, until a model with an appropriate model terms that gives satisfactory fit to the data is obtained.…”

“…The selection procedure can be terminated when specific conditions are met. The number of model terms to be included in the final model can be determined by several model selection criteria, for example, the Generalised Cross-Validation (GCV) [8], a modified Generalised Cross-Validation Criteria based on Mean-Square-Error (MSE) [22], a modified ESR (Error Signal Ratio) index [28] and the adjustable prediction error sum of squares (APRESS) [6]. In this study, the APRESS is used to determine the number of model terms.…”

“…The estimates of K groups of parameter vector can then be calculated from the triangular equations . The final model parameter estimation is chosen to be the average of the parameter estimates, with: (28) Detailed derivation and explanation for the mechanism of the above calculations (25)-(28) can be found in [14] and [15].…”

A wavelet neural network model for spatio-temporal image processing and modeling

“…The initial full model is in general highly redundant. A forward orthogonal regression learning algorithm [13]- [15], implemented using a mutual information method, is then applied to refine and improve the initial full model by removing redundant basis functions.…”

“…The initial full model thus needs to be refined. The FOR-MI algorithm was performed, over the given training dataset, to select significant individual basis functions from the initial model (15). An adjustable generalized cross-validation (readers are referred to [15] for the definition of the AGCV) suggests that a total of 26 basis functions should be included in the final model.…”