Model-structure selection by cross-validation

Stoica, Petre; Eykhoff, P.; Janssen, Peter H.; Söderström, Torsten

doi:10.1080/00207178608933575

Cited by 105 publications

(41 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Model selection criteria are often established on the basis of estimates of prediction errors, by inspecting how the identified model performs on future (never used) data sets. One general routine for model selection, which tries to avoid or reduce any possible bias introduced by relying on any particular test data sets, is cross validation (Stone 1974, Stoica et al 1986). Cross-validation has a number of variations, two commonly used variants of which are the leave-one-out (LOO), also called predicted sum of squares (PRESS) (Allen 1974), and generalised cross-validation (GCV) (Craven andWahba 1979, Golub et al 1979).…”

Section: Model Length Determinationmentioning

confidence: 99%

An adaptive orthogonal search algorithm for model subset selection and non-linear system identification

Billings

Wei

2008

International Journal of Control

View full text Add to dashboard Cite

Published paperAbstract: A new adaptive orthogonal search (AOS) algorithm is proposed for model subset selection and nonlinear system identification. Model structure detection is a key step in any system identification problem. This consists of selecting significant model terms from a redundant dictionary of candidate model terms, and determining the model complexity (model length or model size). The final objective is to produce a parsimonious model that can well capture the inherent dynamics of the underlying system. In the new AOS algorithm, a modified generalized cross-validation criterion, called the adjustable prediction error sum of squares (APRESS), is introduced and incorporated into a forward orthogonal search procedure. The main advantage of the new AOS algorithm is that the mechanism is simple and the implementation is direct and easy, and more importantly it can produce efficient model subsets for most nonlinear identification problems.

show abstract

Section: Model Length Determinationmentioning

confidence: 99%

An adaptive orthogonal search algorithm for model subset selection and non-linear system identification

Billings

Wei

2008

International Journal of Control

View full text Add to dashboard Cite

show abstract

“…This, however, would require a priori knowledge of the process. However, a simpler approach relies on the incorporation of the cross-validation principle [65,64] to automate this selection. In relation to PCA, cross-validation has been proposed as a technique to determine the number of retained principal components by Wold [79] and Krzanowski [39].…”

Section: Disjunct Regionsmentioning

confidence: 99%

Developments and Applications of Nonlinear Principal Component Analysis – a Review

Krüger

Zhang

Xie

2008

Lecture Notes in Computational Science and Enginee

View full text Add to dashboard Cite

Summary.Although linear principal component analysis (PCA) originates from the work of Sylvester [67] and Pearson [51], the development of nonlinear counterparts has only received attention from the 1980s. Work on nonlinear PCA, or NLPCA, can be divided into the utilization of autoassociative neural networks, principal curves and manifolds, kernel approaches or the combination of these approaches. This article reviews existing algorithmic work, shows how a given data set can be examined to determine whether a conceptually more demanding NLPCA model is required and lists developments of NLPCA algorithms. Finally, the paper outlines problem areas and challenges that require future work to mature the NLPCA research field. IntroductionPCA is a data analysis technique that relies on a simple transformation of recorded observation, stored in a vector z ∈ R N , to produce statistically independent score variables, stored in t ∈ R n , n ≤ N :(1.1)Here, P is a transformation matrix, constructed from orthonormal column vectors. Since the first applications of PCA [21], this technique has found its way into a wide range of different application areas, for example signal processing [75], factor analysis [29,44], system identification [77], chemometrics [20,66] and more recently, general data mining [11,70,58] including image processing [17,72] and pattern recognition [47,10], as well as process 2 U. Kruger, J. Zhang, and L. Xie monitoring and quality control [1,82] including multiway [48], multiblock [52] and multiscale [3] extensions. This success is mainly related to the ability of PCA to describe significant information/variation within the recorded data typically by the first few score variables, which simplifies data analysis tasks accordingly. Sylvester [67] formulated the idea behind PCA, in his work the removal of redundancy in bilinear quantics, that are polynomial expressions where the sum of the exponents are of an order greater than 2, and Pearson [51] laid the conceptual basis for PCA by defining lines and planes in a multivariable space that present the closest fit to a given set of points. Hotelling [28] then refined this formulation to that used today. Numerically, PCA is closely related to an eigenvector-eigenvalue decomposition of a data covariance, or correlation matrix and numerical algorithms to obtain this decomposition include the iterative NIPALS algorithm [78], which was defined similarly by Fisher and MacKenzie earlier on [80], and the singular value decomposition. Good overviews concerning PCA are given in Mardia et al. [45], Joliffe [32], Wold et al. [80] and Jackson [30].The aim of this article is to review and examine nonlinear extensions of PCA that have been proposed over the past two decades. This is an important research field, as the application of linear PCA to nonlinear data may be inadequate [49]. The first attempts to present nonlinear PCA extensions include a generalization, utilizing a nonmetric scaling, that produces a nonlinear optimization problem [42] and constructing a curves...

show abstract

“…To the best of our knowledge no such criterion has been rigorously developed, and adaptation of existing solutions that were derived for 1-D data models may be misleading. Stoica et al, [22] proposed a cross-validation selection rule and demonstrated its asymptotic equivalence to the Generalized Akaike Information Criterion (GAIC). The suggested criterion is not derived for any specific model.…”

Section: Introductionmentioning

confidence: 99%