A randomized algorithm for nonlinear model structure selection

Abstract

The identification of polynomial Nonlinear Autoregressive [Moving Average] models with eXogenous variables (NAR[MA]X) is typically carried out with incremental model building techniques that progressively select the terms to include in the model. The Model Structure Selection (MSS) turns out to be the hardest task of the identification process due to the difficulty of correctly evaluating the importance of a generic term. As a result, classical MSS methods sometimes yield unsatisfactory models, that are unreliable over long-range prediction horizons. The MSS problem is here recast into a probabilistic framework based on which a randomized algorithm for MSS is derived, denoted RaMSS. The method introduces a tentative probability distribution over models and progressively updates it by extracting useful information on the importance of each term from sampled model structures. The proposed method is validated over models with different characteristics by means of Monte Carlo simulations, which show its advantages over classical and competitor probabilistic MSS methods in terms of both reliability and computational efficiency

Multi-objective evolutionary framework for non-linear system identification: A comprehensive investigation

“…where, 'J(·)' denotes a suitable criterion function. Most of the existing structure selection approaches often use the predictive performance of the model as the criterion function [28][29][30][31][32][33][34]. These can, therefore, be categorized as the single objective approach to structure selection.…”

“…In such scenarios, the information criteria usually tend to over-fit as suggested by the earlier investigation of Nakamura et al [27]. Hence, the estimation of cardinality still requires further investigations in OFR based approaches.The other interesting approach to structure selection is based on stochastic sampling of the search space, e.g., Evolutionary Algorithms [28][29][30][31][32][33][34] and Bayesian inference [35,36]. In the earlier investigations, Genetic Algorithm (GA) [28,29] and Genetic Programming (GP) [30,31] have been proposed as an alternative to OFR.…”

“…In these investigations, the structure selection has been posed as a single objective problem, where the search objective is chosen to be an aggregated function of predictive performance and number of terms to balance the bias-variance dilemma. Recently, a randomized structure selection approach is proposed in [32][33][34]. In this approach, each candidate term is assigned with selection probability which is updated following a positive-feedback mechanism similar to ant colony optimization [37].…”

Identification of robotic systems with hysteresis using Nonlinear AutoRegressive eXogenous input models

“…Several criteria have been proposed in the literature for NARX models. 25 One of the most widely used is the error reduction ratio (ERR) based on the OFR algorithm. 26 The authors of this article also use it as a reference of the final chosen term detection method.…”

Computational system identification of continuous-time nonlinear systems using approximate Bayesian computation

“…Characterisation of uncertainty is important in control engineering (Gevers, 2005), but also in other areas where SID is now commonly applied such as the life sciences (Anderson, Lepora, Porrill, & Dean, 2010;Krishnanathan, Anderson, Billings, & Kadirkamanathan, 2012;Kukreja, Galiana, & Kearney, 2003). In SID, computational Bayesian (or probabilistic) methods are gaining popularity due to advances in processing power (Baldacchino, Anderson, & Kadirkamanathan, 2013;Falsone, Piroddi, & Prandini, 2015;Ninness & Henriksen, 2010). The computational estimation framework we develop here for CT-SID is based on approximate Bayesian computation (ABC), which is a rejection sampling algorithm (Beaumont, Zhang, & Balding, 2002;Tavare, Balding, Griffiths, & Donnelly, 1997).…”