Benchmarking sparse system identification with low-dimensional chaos

Kaptanoglu, Alan A.; Zhang, Lanyue; Nicolaou, Zachary G.; Fasel, Urban; Brunton, Steven L.

doi:10.1007/s11071-023-08525-4

Cited by 8 publications

(2 citation statements)

References 118 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, we employ the WSINDy algorithm, which has its roots in the SINDy algorithm 17 . The weak form has risen to prominence as a way to combat realistic challenges like noisy data and non-smooth dynamics [18][19][20][21][22][23][24][25][26][27][28][29][30] . Most relevant to this work, WSINDy has been demonstrated to offer coarse-graining capabilities 31 in the context of interacting particle systems and homogenization of parabolic PDEs, and more recently in reduced order modeling applications 32 .…”

Section: Literature Reviewmentioning

confidence: 99%

Coarse-graining Hamiltonian systems using WSINDy

Messenger,

Burby,

Bortz

2024

Sci Rep

View full text Add to dashboard Cite

Weak form equation learning and surrogate modeling has proven to be computationally efficient and robust to measurement noise in a wide range of applications including ODE, PDE, and SDE discovery, as well as in coarse-graining applications, such as homogenization and mean-field descriptions of interacting particle systems. In this work we extend this coarse-graining capability to the setting of Hamiltonian dynamics which possess approximate symmetries associated with timescale separation. A smooth $$\varepsilon$$ ε -dependent Hamiltonian vector field $$X_\varepsilon$$ X ε possesses an approximate symmetry if the limiting vector field $$X_0=\lim _{\varepsilon \rightarrow 0}X_\varepsilon$$ X 0 = lim ε → 0 X ε possesses an exact symmetry. Such approximate symmetries often lead to the existence of a Hamiltonian system of reduced dimension that may be used to efficiently capture the dynamics of the symmetry-invariant dependent variables. Deriving such reduced systems, or approximating them numerically, is an ongoing challenge. We demonstrate that WSINDy can successfully identify this reduced Hamiltonian system in the presence of large perturbations imparted in the $$\varepsilon >0$$ ε > 0 regime, while remaining robust to extrinsic noise. This is significant in part due to the nontrivial means by which such systems are derived analytically. WSINDy naturally preserves the Hamiltonian structure by restricting to a trial basis of Hamiltonian vector fields. The methodology is computationally efficient, often requiring only a single trajectory to learn the global reduced Hamiltonian, and avoiding forward solves in the learning process. In this way, we argue that weak-form equation learning is particularly well-suited for Hamiltonian coarse-graining. Using nearly-periodic Hamiltonian systems as a prototypical class of systems with approximate symmetries, we show that WSINDy robustly identifies the correct leading-order system, with dimension reduced by at least two, upon observation of the relevant degrees of freedom. While our main contribution is computational, we also provide a contribution to the literature on averaging theory by proving that first-order averaging at the level of vector fields preserves Hamiltonian structure in nearly-periodic Hamiltonian systems. This provides theoretical justification for our approach as WSINDy’s computations occur at the level of Hamiltonian vector fields. We illustrate the efficacy of our proposed method using physically relevant examples, including coupled oscillator dynamics, the Hénon–Heiles system for stellar motion within a galaxy, and the dynamics of charged particles.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Coarse-graining Hamiltonian systems using WSINDy

Messenger,

Burby,

Bortz

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…The weak form has risen to prominence as a way to combat realistic challenges like noisy data and non-smooth dynamics [18][19][20][21][22][23][24][25][26][27][28][29][30]. Most relevant to this work, WSINDy has been demonstrated to offer coarse-graining capabilities [31] in the context of interacting particle systems and homogenization of parabolic PDEs.…”

Section: Literature Reviewmentioning

confidence: 99%

Coarse-Graining Hamiltonian Systems Using WSINDy

Messenger,

Burby,

Bortz

2023

Preprint

View full text Add to dashboard Cite

The Weak-form Sparse Identification of Nonlinear Dynamics algorithm (WSINDy) has been demonstrated to offer coarse-graining capabilities in the context of interacting particle systems (https://doi.org/10.1016/j.physd.2022.133406). In this work we extend this capability to the problem of coarse-graining Hamiltonian dynamics which possess approximate symmetries associated with timescale separation. A smooth ε-dependent Hamiltonian vector field Xε possesses an approximate symmetry if the limiting vector field X0 = limε→0 Xε possesses an exact symmetry. Such approximate symmetries often lead to the existence of a Hamiltonian system of reduced dimension that may be used to efficiently capture the dynamics of the symmetry-invariant dependent variables. Deriving such reduced systems, or approximating them numerically, is an ongoing challenge. We demonstrate that WSINDy can successfully identify this reduced Hamiltonian system in the presence of large perturbations imparted in the ε > 0 regime, while remaining robust to extrinsic noise. This is significant in part due to the nontrivial means by which such systems are derived analytically. WSINDy naturally preserves the Hamiltonian structure by restricting to a trial basis of Hamiltonian vector fields. The methodology is computational efficient, often requiring only a single trajectory to learn the global reduced Hamiltonian, and avoiding forward solves in the learning process. In this way, we argue that weak-form equation learning is particularly well-suited for Hamiltonian coarse-graining. Using nearly-periodic Hamiltonian systems as a prototypical class of systems with approximate symmetries, we show that WSINDy robustly identifies the correct leading-order system, with dimension reduced by at least two, upon observation of the relevant degrees of freedom. We also provide a contribution to the literature on averaging theory by proving that first-order averaging at the level of vector fields preserves Hamiltonian structure in nearly-periodic Hamiltonian systems. We provide physically relevant examples, namely coupled oscillator dynamics, the Hénon-Heiles system for stellar motion within a galaxy, and the dynamics of charged particles.

show abstract

Detach-ROCKET: sequential feature selection for time series classification with random convolutional kernels

Uribarri,

Barone,

Ansuini

et al. 2024

Data Min Knowl Disc

View full text Add to dashboard Cite

Time Series Classification (TSC) is essential in fields like medicine, environmental science, and finance, enabling tasks such as disease diagnosis, anomaly detection, and stock price analysis. While machine learning models like Recurrent Neural Networks and InceptionTime are successful in numerous applications, they can face scalability issues due to computational requirements. Recently, ROCKET has emerged as an efficient alternative, achieving state-of-the-art performance and simplifying training by utilizing a large number of randomly generated features from the time series data. However, many of these features are redundant or non-informative, increasing computational load and compromising generalization. Here we introduce Sequential Feature Detachment (SFD) to identify and prune non-essential features in ROCKET-based models, such as ROCKET, MiniRocket, and MultiRocket. SFD estimates feature importance using model coefficients and can handle large feature sets without complex hyperparameter tuning. Testing on the UCR archive shows that SFD can produce models with better test accuracy using only 10% of the original features. We named these pruned models Detach-ROCKET. We also present an end-to-end procedure for determining an optimal balance between the number of features and model accuracy. On the largest binary UCR dataset, Detach-ROCKET improves test accuracy by 0.6% while reducing features by 98.9%. By enabling a significant reduction in model size without sacrificing accuracy, our methodology improves computational efficiency and contributes to model interpretability. We believe that Detach-ROCKET will be a valuable tool for researchers and practitioners working with time series data, who can find a user-friendly implementation of the model at https://github.com/gon-uri/detach_rocket.

show abstract

Benchmarking sparse system identification with low-dimensional chaos

Cited by 8 publications

References 118 publications

Coarse-graining Hamiltonian systems using WSINDy

Coarse-graining Hamiltonian systems using WSINDy

Coarse-Graining Hamiltonian Systems Using WSINDy

Detach-ROCKET: sequential feature selection for time series classification with random convolutional kernels

Contact Info

Product

Resources

About