Neural closure models for dynamical systems

Abstract

Complex dynamical systems are used for predictions in many domains. Because of computational costs, models are truncated, coarsened or aggregated. As the neglected and unresolved terms become important, the utility of model predictions diminishes. We develop a novel, versatile and rigorous methodology to learn non-Markovian closure parametrizations for known-physics/low-fidelity models using data from high-fidelity simulations. The new neural closure models augment low-fidelity models with neural delay differential equations (nDDEs), motivated by the Mori–Zwanzig formulation and the inherent delays in complex dynamical systems. We demonstrate that neural closures efficiently account for truncated modes in reduced-order-models, capture the effects of subgrid-scale processes in coarse models and augment the simplification of complex biological and physical–biogeochemical models. We find that using non-Markovian over Markovian closures improves long-term prediction accuracy and requires smaller networks. We derive adjoint equations and network architectures needed to efficiently implement the new discrete and distributed nDDEs, for any time-integration schemes and allowing non-uniformly spaced temporal training data. The performance of discrete over distributed delays in closure models is explained using information theory, and we find an optimal amount of past information for a specified architecture. Finally, we analyse computational complexity and explain the limited additional cost due to neural closure models.

Reduced basis methods for time-dependent problems

“…One way to look at this problem is as a closure R -269 problem, akin to that found in turbulence modelling. This point of view has led to recent work focusing on the formulation of closure models for time-dependent reduced order models, based on the Mori-Zwanzig formulation (Gouasmi, Parish, andDuraisamy 2017, Wang, Ripamonti and, delay-differential equations (Gupta and Lermusiaux 2021), and a substantial effort for closure models inspired by developments in fluid dynamics, reviewed by Ahmed et al (2021). This line of work is still at an early stage of development but these initial results, even if largely of a heuristic and problem-dependent nature, offer some interesting ideas for future work to improve the stability of reduced models and, as demonstrated in , enable the development of reduced order models with predictive accuracy beyond the training interval.…”

A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic

“…The ML is trained to correct the results of the physical model by using the output of the real system as the target [313,358]. Similarly, in Residual Modelling, ML learns to model the PDM error, therefore the ML can correct the PDM output or classify its validity, as in [354,[359][360][361][362][363]. Finally, the ML can be used just as a sub-process of the PDM to evaluate one of its parameters [95,[364][365][366][367][368].…”

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

“…Consequently, an emerging thrust in modern ROM closure development efforts is to incorporate machine learning (ML) models 4,[8][9][10][11][12] . The last decade has seen the growth of data-driven modeling technologies (e.g., deep neural networks).…”