Model structure selection using an integrated forward orthogonal search algorithm assisted by squared correlation and mutual information

Abstract

Published paperMappin Street, Sheffield, S1 3JD, UK Abstract: Model structure selection plays a key role in nonlinear system identification. The first step in nonlinear system identification is to determine which model terms should be included in the model. Once significant model terms have been determined, a model selection criterion can then be applied to select a suitable model subset. The well known orthogonal least squares type algorithms are one of the most efficient and commonly used techniques for model structure selection. However, it has been observed that the orthogonal least squares type algorithms may occasionally select incorrect model terms or yield a redundant model subset in the presence of particular noise structures or input signals. A very efficient integrated forward orthogonal search (IFOS) algorithm, which is assisted by the squared correlation and mutual information, and which incorporates a generalised cross-validation (GCV) criterion and hypothesis tests, is introduced to overcome these limitations in model structure selection.

Time-varying parametric modelling and time-dependent spectral characterisation with applications to EEG signals using multiwavelets

“…The well-known orthogonal least squares (OLS) type of algorithms Aguirre and Billings, 1995;Zhu and Billings, 1996;Wei and Billings, 2008) have been proven to be very effective to deal with multiple dynamical regression problems, which involve a great number of candidate model terms or regressors that may be highly correlated. In the present study, the OLS algorithm given in , is used to solve the regression equation (4).…”

GENERALIZED CELLULAR NEURAL NETWORKS (GCNNs) CONSTRUCTED USING PARTICLE SWARM OPTIMIZATION FOR SPATIO-TEMPORAL EVOLUTIONARY PATTERN IDENTIFICATION

“…Similar to other constructive algorithms, models produced by the OPP algorithm may, however, be highly redundant. To remove or reduce redundancy, a forward orthogonal regression (FOR) learning algorithm [Billings & Wei, 2007a;Wei & Billings, 2007], implemented using a mutual information estimation method, is applied to refine and improve the initially generated model by the OPP algorithm.…”

“…However, like the conventional projection pursuit regression algorithm, the OPP algorithm may produce redundant models. To refine and improve the OPP produced network models, the forward orthogonal regression (FOR) learning algorithm, assisted by a mutual information method [Billings & Wei, 2007a;Wei & Billings, 2007], is then applied to remove any severe redundancy. …”

“…In this study, however, a variation of the OLS algorithm, called the forward orthogonal regression (FOR) algorithm, implemented using a mutual information method [Billings & Wei, 2007a;Wei & Billings, 2007], is employed for the model refinement. Assume that x and y are two random discrete variables, with alphabet X and , respectively, and with a joint probability mass function p(x, y) and marginal probability mass functions and .…”

“…Detailed discussions on the utility of the mutual information for model term selection can be found in Billings and Wei [2007a] and Wei and Billings [2007]. Now, let be the n selected…”