Koopman Operator Family Spectrum for Nonautonomous Systems

Mačešić, Senka; Črnjarić-Žic, Nelida; Mezić, Igor

doi:10.1137/17m1133610

Cited by 33 publications

(24 citation statements)

References 31 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, no such method exists for KMD. And fourth, whether and how our results change when the underlying dynamical system is non-autonomous, an area of recent active research in the KOT community 63 , will be the direction of future work.…”

Section: Discussionmentioning

confidence: 95%

On Koopman Mode Decomposition and Tensor Component Analysis

Redman

2021

Preprint

View full text Add to dashboard Cite

Koopman mode decomposition and tensor component analysis are two tools that decompose high dimensional data sets into low dimensional modes that capture relevant features and/or dynamics. Despite their similar goal, the two methods are largely used by distinct scientific communities. For the first time, examine the two together and show that, under a certain (reasonable) condition on the data, the theoretical decomposition given by tensor component analysis is the same as that given by Koopman mode decomposition. This realization provides possibilities for new algorithmic approaches to both Koopman mode decomposition and tensor component analysis, new insight into what is being captured by the tensor component analysis modes, and a "bridge" with which the two communities can more effectively communicate.Koopman operator theory (KOT) has emerged as a powerful and general framework with which to understand nonlinear dynamical systems in a data-driven manner. Despite its many strengths, there are numerous scientific fields, such as neuroscience and biology, where it remains under employed. As part of this stems from KOT being formulated mathematically in a distinct way from existing, commonly used methods, such as principal component analysis (PCA), there is a need for bridging KOT with such methods. Here, for the first time, we investigate how a major KOT analysis method differs from tensor component analysis (TCA), an extension of PCA that has quickly become adopted by some of those same fields hesitant on KOT. We show that, in certain scenarios, TCA and KOT, in theory, give the same decomposition of data. This not only makes strides in establishing a bridge between TCA and KOT, but also provides new insight into TCA and new possible algorithms for implementing KOT and TCA.

show abstract

Section: Discussionmentioning

confidence: 95%

On Koopman Mode Decomposition and Tensor Component Analysis

Redman

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this section, we present two algorithms, namely weighted incremental DMD and windowed incremental DMD, respectively, for incremental computation of a time-varying lower-dimensional DMD operator. Similar to the existing approaches in the literature, the proposed algorithms perform incremental computations either by assigning decaying weights to the received data [16,35], or by using a sliding window of the received data [10,20,35]. The recursive updates derived in this section are similar to those in sequential least square method [16, Appendix 8C], recursive least square method [15, Section 9.4], as well as online DMD [35].…”

Section: 1mentioning

confidence: 99%

An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling

Alfatlawi¹,

Srivastava²

2020

Journal of Computational Dynamics

View full text Add to dashboard Cite

Dynamic Mode Decomposition (DMD) is a data-driven technique to identify a low dimensional linear time invariant dynamics underlying highdimensional data. For systems in which such underlying low-dimensional dynamics is time-varying, a time-invariant approximation of such dynamics computed through standard DMD techniques may not be appropriate. We focus on DMD techniques for such time-varying systems and develop incremental algorithms for systems without and with exogenous control inputs. We build upon the work in [35] to scenarios in which high dimensional data are governed by low dimensional time-varying dynamics. We consider two classes of algorithms that rely on (i) a discount factor on previous observations, and (ii) a sliding window of observations. Our algorithms leverage existing techniques for incremental singular value decomposition and allow us to determine an appropriately reduced model at each time and are applicable even if data matrix is singular. We apply the developed algorithms for autonomous systems to Electroencephalographic (EEG) data and demonstrate their effectiveness in terms of reconstruction and prediction. Our algorithms for non-autonomous systems are illustrated using randomly generated linear time-varying systems.

show abstract

“…T is not an explicit function of t). However, generalizations to non-autonomous systems are available [31].…”

Section: Kot and Its Connection To Nnsmentioning

confidence: 99%

Optimizing Neural Networks via Koopman Operator Theory

Dogra¹,

Redman²

2020

Preprint

View full text Add to dashboard Cite

Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in making use of this connection. As Koopman operator theory is a linear theory, a successful implementation of it in evolving network weights and biases offers the promise of accelerated training, especially in the context of deep networks, where optimization is inherently a non-convex problem. We show that Koopman operator theory methods allow for accurate predictions of the weights and biases of a feedforward, fully connected deep network over a non-trivial range of training time. During this time window, we find that our approach is at least 10x faster than gradient descent based methods, in line with the results expected from our complexity analysis. We highlight additional methods by which our results can be expanded to broader classes of networks and larger time intervals, which shall be the focus of future work in this novel intersection between dynamical systems and neural network theory. IntroductionDespite their black box nature, the training of artificial neural networks (NNs) is a discrete dynamical system. During training, NN weights evolve along a trajectory in an abstract weight space, the path determined by the implemented learning algorithm, the data used for training, and the network architecture. This dynamical systems picture is familiar, as many introductions to learning algorithms, such as gradient descent (GD), attempt to visualize training as a process whereby weights are changed iteratively under the influence of the loss landscape. Yet, while dynamical systems theory has provided insight into the behavior of many complex systems, its application to NNs has been limited.Recent advances in Koopman operator theory (KOT) have made it a powerful tool in studying the underlying dynamics of nonlinear systems in a data-driven manner [1][2][3][4][5][6][7][8][9]. This begs the question, can KOT be used to learn and predict the dynamics present in NN training? If so, can such an approach, which we call Koopman training, afford us benefits that traditional NN training methods cannot? * The authors contributed equally Preprint. Under review.

show abstract

Koopman Operator Family Spectrum for Nonautonomous Systems

Cited by 33 publications

References 31 publications

On Koopman Mode Decomposition and Tensor Component Analysis

On Koopman Mode Decomposition and Tensor Component Analysis

An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling

Optimizing Neural Networks via Koopman Operator Theory

Contact Info

Product

Resources

About