Predicting Physics in Mesh-reduced Space with Temporal Attention

Abstract

Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.

Integrating Graph Neural Network-Based Surrogate Modeling with Inverse Design for Granular Flows

“…Looking ahead, the potential applications of GNS in industrial and research settings are extensive, especially in fields requiring rapid and accurate simulations of granular material behavior. Future work could focus on enhancing the GNS model training efficiency, perhaps by exploring more advanced machine learning techniques or hybrid models to handle time sequence information on the trajectories more efficiently. − Further research could also delve into expanding the model applicability to three-dimensional simulations and more complex material interactions associated with polydisperse particle systems. Additionally, the model could be enhanced by incorporating a broader range of material properties like particle packing fraction, nonspherical particle shapes, and complex domain boundaries …”

Graph convolutional multi-mesh autoencoder for steady transonic aircraft aerodynamics

“…The following properties are desirable for an adequate interpolation [35]: (1) interpolated values at the source nodes should match the original data; (2) integrated resultants should be conserved; and (3) interpolated fields should be continuous. Consequently, directly recasting the data across coincident points and nearest-neighbour interpolation, as Han et al [20] proposed, result inappropriate because conservation and continuity properties are not satisfied.…”

“…Baqué et al [18] projected the 3D geometry to a simpler prismatic graph to contain the computational burden. On the other hand, comprehensive message-passing methods, as adopted by Hines and Bekemeyer [19] or Han et al [20], which feature dedicated learnable weights for each edge and node of the mesh, can lead to excessive memory requirements.…”

Machine Learning enabled Wireless Communication Network System

“…The Fourier Neural Operator (FNO) (Li et al, 2020) and its subsequent iterations (Li et al, 2021;Guibas et al, 2021;Tran et al, 2021) have demonstrated their efficacy in a variety of contexts. Building on the momentum of attention mechanisms (Vaswani et al, 2017), numerous studies (Geneva & Zabaras, 2022;Kissas et al, 2022;Han et al, 2022) have utilized attention to model and simulate physical phenomena. The Galerkin Transformer (Cao, 2021), for instance, employs attention layers to discern the underlying structures and patterns within the spatial domain of PDEs.…”