Using the minimum description length principle for global reconstruction of dynamic systems from noisy time series

Abstract

An alternative approach to determining embedding dimension when reconstructing dynamic systems from a noisy time series is proposed. The available techniques of determining embedding dimension (the false nearest-neighbor method, calculation of the correlation integral, and others) are known [H. D. I. Abarbanel, (Springer-Verlag, New York, 1997)] to be inefficient, even at a low noise level. The proposed approach is based on constructing a global model in the form of an artificial neural network. The required amount of neurons and the embedding dimension are chosen so that the description length should be minimal. The considered approach is shown to be appreciably less sensitive to the level and origin of noise, which makes it also a useful tool for determining embedding dimension when constructing stochastic models.

A Representational MDL Framework for Improving Learning Power of Neural Network Formalisms

“…This fact is typically ignored because of the well-known proof that ANNs are universal approximators [20]. This proof is cited even in the papers devoted to the MDL-based ANNs [17]. This contradiction arises from a lack of understanding that approximation of any function with preset accuracy is insufficient in machine learning.…”

“…In particular, the MDL principle was applied in this heuristic way to solve the problem of ANN architecture optimization [15][16][17]. In these works, components of the description length are calculated within some ANN coding schemes.…”

Self-organizing kernel adaptive filtering

“…After that, as an improvement, the MDL criterion was utilized to directly determine an optimal neural network model and successfully applied to prediction problems [36] and control systems [37]. Furthermore, the embedding dimension of an artificial neural network is decided based on constructing a global model with a least description length [38]. Starting from an overly complex model and then pruning unneeded basis function according to MDL, Leonardis and Bischof [39] proposed a radial basis function (RBF) network formulation to balance accuracy performance, training time, and network complexity.…”

Predicting Critical Transitions in ENSO models. Part II: Spatially Dependent Models

“…In this paper, we apply the empirical stochastic model of Molkov et al (2012) and Part I to the prediction of critical transitions in the climate system, when weakly nonstationary time series-which are spatially distributed, rather than merely scalar-are available. As a step toward the use of actually observed climate data, we worked with data from an intermediate-complexity, hybrid coupled ENSO model (Neelin 1991;Jin and Neelin 1993a) over the tropical Pacific Ocean.…”

“…In two previous papers (Molkov et al 2012;Mukhin et al 2015, hereinafter Part I), we formulated an empirical modeling and prediction methodology based on artificial neural networks (ANNs; Hornik et al 1989). A key difficulty in applying this methodology to construct an empirical, nonlinear stochastic model that helps simulate and predict the real climate system's behavior is the mismatch between the large number of variables by which one wishes to describe the system versus the shortness of the time series of available experimental data.…”