Leveraging information storage to select forecast-optimal parameters for delay-coordinate reconstructions

Abstract

Delay-coordinate reconstruction is a proven modeling strategy for building effective forecasts of nonlinear time series. The first step in this process is the estimation of good values for two parameters, the time delay and the embedding dimension. Many heuristics and strategies have been proposed in the literature for estimating these values. Few, if any, of these methods were developed with forecasting in mind, however, and their results are not optimal for that purpose. Even so, these heuristics-intended for other applications-are routinely used when building delay coordinate reconstruction-based forecast models. In this paper, we propose an alternate strategy for choosing optimal parameter values for forecast methods that are based on delay-coordinate reconstructions. The basic calculation involves maximizing the shared information between each delay vector and the future state of the system. We illustrate the effectiveness of this method on several synthetic and experimental systems, showing that this metric can be calculated quickly and reliably from a relatively short time series, and that it provides a direct indication of how well a near-neighbor based forecasting method will work on a given delay reconstruction of that time series. This allows a practitioner to choose reconstruction parameters that avoid any pathologies, regardless of the underlying mechanism, and maximize the predictive information contained in the reconstruction.

Criticality and Information Dynamics in Epidemiological Models

“…In order to determine which embedding length k is most suitable for our analysis, we seek to set k so as to maximise the average active information storage, as per the criteria presented in [39]. Importantly though, for this criteria to work we need to maximise the bias-corrected active information storage rather than its raw value.…”

Transitions in brain-network level information processing dynamics are driven by alterations in neural gain

“…The determination of these embedding parameters followed the method of Garland et al [79] finding the values which maximize the AIS, with the important additional inclusion of bias correction (because increasing k generally serves to increase bias of the estimate) [80]. For several sample σ, γ pairs in both the sub-and supercritical regimes we examined these parameter choices across all regions (up to k, τ ≤ 30), and found the optimal choices to be consistently close to k = 25 and τ = 12 for all variables (for the sampling interval ∆t = 0.5 ms).…”

Characterizing information-theoretic storage and transfer in continuous time processes

“…One of the more striking corollaries of the preceding formalism is that purely Markov processes like the Ornstein-Uhlenbeck process in Eq. (14), whilst having non-zero, and indeed divergent, active information storage rates, have a vanishing memory utilisation rate since, by definition in their construction, they only have dependence on their most recent state. In many respects this is appealing as the memory utilisation rate then aligns very closely with the intuitive definition of a Markov process as being 'memoryless'.…”

“…a simple Ornstein-Uhlenbeck process as in Eq. (14), the contributions to the instantaneous predictive capacity, I X , and components I I X andİ R X , are equal to those in Eq. (30), but with κ = κ eff X from Eq.…”

“…These primitive information processing operations studied by the framework, information storage and transfer, are measured by the active information storage [5] and transfer entropy [6] (and higher orders thereof in larger multivariate systems [4,7]) respectively. The measures in this framework have been used to characterise and explain behaviour observed in various complex systems, providing novel insights across a broad range of fields including in: canonical complex systems such as cellular automata [7] and random Boolean networks [8], interpretation of dynamics in [9] and improved algorithms for machine learning [10], characterising information processing signatures in biological signalling networks [11], collective behaviour in swarms [12,13], non-linear timeseries forecasting [14], and in computational neuroscience applications in identifying neural information flows from brain imaging data [15][16][17], inferring effective network structure [18][19][20], providing evidence for the predictive coding hypothesis [21] and identifying differences in predictive information in autism spectrum disorder subjects and controls [22]. However, to date, no complete theoretical account for how this framework should be applied to continuous time systems have been offered, despite such systems being ubiquitous in fields throughout all of science.…”