Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-Valued Observables

Giannakis, Dimitrios; Ourmazd, A.; Sławińska, Joanna; Zhao, Zhizhen

doi:10.1007/s00332-019-09548-1

Cited by 27 publications

(32 citation statements)

References 72 publications

(219 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, }, (i) the eigenvalues λ i,n converge to λ i ; (ii) the RKHS functions ψ i,n converge, up to multiplication by a constant phase factor, to ψ i in C(U) norm; and (iii) each of the expansion coefficients α i,n (τ) converges to α i (τ). The first two of these claims are a consequence of the following lemma, which is based on [46,Theorem 15], [55,Corollary 2], and [56,Theorem 7].…”

Section: Kaf Sample Errormentioning

confidence: 99%

Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques

Alexander

Giannakis

2020

Physica D: Nonlinear Phenomena

Self Cite

View full text Add to dashboard Cite

Kernel analog forecasting (KAF), alternatively known as kernel principal component regression, is a kernel method used for nonparametric statistical forecasting of dynamically generated time series data. This paper synthesizes descriptions of kernel methods and Koopman operator theory in order to provide a single consistent account of KAF. The framework presented here illuminates the property of the KAF method that, under measure-preserving and ergodic dynamics, it consistently approximates the conditional expectation of observables that are acted upon by the Koopman operator of the dynamical system and are conditioned on the observed data at forecast initialization. More precisely, KAF yields optimal predictions, in the sense of minimal root mean square error with respect to the invariant measure, in the asymptotic limit of large data. The presented framework facilitates, moreover, the analysis of generalization error and quantification of uncertainty. Extensions of KAF to the construction of conditional variance and conditional probability functions are also shown. Illustrations of various aspects of KAF are provided with applications to simple examples, namely a periodic flow on the circle and the chaotic Lorenz 63 system. (Dimitrios Giannakis) science applications, abstract statistical structures are the focus when situated in a more general machine learning context. Common nonparametric machine learning techniques include multilayer perceptrons [12], Bayesian neural networks [13], classification and regression trees (CART), and a variety of kernel methods [14]. Although each of these methods can provide value in unique ways to specific problems, kernel methods are particularly well suited to problems where there may be a natural, a priori, notion of similarity between data points. Since analog methods rely on the possibility that the relevance of any historical analog to present day conditions can be quantitatively determined, formal understanding of such methods can improve when they are cast within the larger framework of kernel methods.Kernel methods constitute a class of algorithms that perform classical calculations in a rich functional feature space in order to extract and predict nonlinear patterns. This central idea, commonly referred to as "the kernel trick", was first proposed in 1964 [15], was popularized with the invention of nonlinear support vector machines (SVMs) in 1992 [16], and has since spread to a variety of machine learning applica-

show abstract

Section: Kaf Sample Errormentioning

confidence: 99%

Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques

Alexander

Giannakis

2020

Physica D: Nonlinear Phenomena

Self Cite

View full text Add to dashboard Cite

show abstract

“…That said, there have been some progress in understanding the RC-ESNs, in particular by modeling the network itself as a dynamical system [65,53]. Such efforts, for example those aimed at understanding the echo states that are learned in the RC-ESN's reservoir, might benefit from recent advances in dynamical systems theory [66,67,68,69,70,71].…”

Section: Discussionmentioning

confidence: 99%

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Chattopadhyay¹,

Hassanzadeh²,

Subramanian³

2020

Preprint

View full text Add to dashboard Cite

Abstract. In this paper, the performance of three deep learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multi-scale spatio-temporal Lorenz 96 system is examined. The methods are: echo state network (a type of reservoir computing, RC-ESN), deep feed-forward artificial neural network (ANN), and recurrent neural network with long short-term memory (RNN-LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC-ESN substantially outperforms ANN and RNN-LSTM for short-term prediction, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps, equivalent to several Lyapunov timescales. The RNN-LSTM and ANN show some prediction skills as well; RNN-LSTM bests ANN. Furthermore, even after losing the trajectory, data predicted by RC-ESN and RNN-LSTM have probability density functions (PDFs) that closely match the true PDF, even at the tails. The PDF of the data predicted using ANN, however, deviates from the true PDF. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems such as weather/climate are discussed.

show abstract

“…The eigenvectors φ j then provide representations of the basis elements φ j,N (see Section V C). Elsewhere, we have demonstrated the feasibility of this implementation for computing eigenfunctions from high-dimensional datasets of moderate sample number, (d, N ) = O(10 6 , 10 4 ) [51], or datasets of moderate dimension and high sample number, (d, N ) = O(10 2 , 10 6 ) [26]. In the latter case, it should be possible to speed up the kernel matrix calculation using tree-based [52] or randomized [53] approximate nearest-neighbor algorithms, though we have not explored such options in the present work.…”

Section: Data-driven Basismentioning

confidence: 98%

“…of L 2 (µ), consisting of eigenfunctions of G µ . See [25,26] for proofs of these results, which make use of spectral convergence results for kernel integral operators established in [13]. Following [25], we construct the kernelsp N starting from an unnormalized kernelk N : M × M → R, and applying to that kernel a normalization procedure to render it Markovian.…”

Section: Kernels and Their Associated Eigenfunction Basesmentioning

confidence: 99%

Quantum mechanics and data assimilation

Giannakis

2019

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

A framework for data assimilation combining aspects of operator-theoretic ergodic theory and quantum mechanics is developed. This framework adapts the Dirac-von Neumann formalism of quantum dynamics and measurement to perform sequential data assimilation (filtering) of a partially observed, measure-preserving dynamical system, using the Koopman operator on the L 2 space associated with the invariant measure as an analog of the Heisenberg evolution operator in quantum mechanics. In addition, the state of the data assimilation system is represented by a trace-class operator analogous to the density operator in quantum mechanics, and the assimilated observables by self-adjoint multiplication operators. An averaging approach is also introduced, rendering the spectrum of the assimilated observables discrete, and thus amenable to numerical approximation. We present a data-driven formulation of the quantum mechanical data assimilation approach, utilizing kernel methods from machine learning and delay-coordinate maps of dynamical systems to represent the evolution and measurement operators via matrices in a data-driven basis. The data-driven formulation is structurally similar to its infinite-dimensional counterpart, and shown to converge in a limit of large data under mild assumptions. Applications to periodic oscillators and the Lorenz 63 system demonstrate that the framework is able to naturally handle highly non-Gaussian statistics, complex state space geometries, and chaotic dynamics.

show abstract

Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-Valued Observables

Cited by 27 publications

References 72 publications

Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques

Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

Quantum mechanics and data assimilation

Contact Info

Product

Resources

About