Building high accuracy emulators for scientific simulations with deep neural architecture search

Kasim, Muhammad; Watson-Parris, Duncan; Deaconu, Lucia; Oliver, Sophy; Hatfield, Peter; Froula, Dustin; Gregori, G.; Jarvis, M. J.; Khatiwala, S.; Korenaga, Jun; Topp-Mugglestone, J.; Viezzer, E.; Vinko, S. M.

doi:10.1088/2632-2153/ac3ffa

Cited by 80 publications

(55 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These authors hence also use the deviation of the network prediction from the model as an additional penalty term in the training loss. Kasim et al [25] recently demonstrated neural network-based simulations for a broad range of applications, including fluid dynamics, by also optimizing the network architecture itself during training. Eichinger, Heinlein and Klawonn [33] use convolutional neural networks and techniques from image processing to learn flow patterns for the Navier-Stokes flow around objects of different shape.…”

Section: Related Workmentioning

confidence: 99%

Structure Preservation for the Deep Neural Network Multigrid Solver

Margenberg,

Lessig,

Richter

2020

Preprint

View full text Add to dashboard Cite

The simulation of partial differential equations is a central subject of numerical analysis and an indispensable tool in science, engineering and related fields. Existing approaches, such as finite elements, provide (highly) efficient tools but deep neural network-based techniques emerged in the last few years as an alternative with very promising results. We investigate the combination of both approaches for the approximation of the Navier-Stokes equations and to what extent structural properties such as divergence freedom can and should be respected. Our work is based on DNN-MG, a deep neural network multigrid technique, that we introduced recently and which uses a neural network to represent fine grid fluctuations not resolved by a geometric multigrid finite element solver. Although DNN-MG provides solutions with very good accuracy and is computationally highly efficient, we noticed that the neural network-based corrections substantially violate the divergence freedom of the velocity vector field. In this contribution, we discuss these findings and analyze three approaches to address the problem: a penalty term to encourage divergence freedom of the network output; a penalty term for the corrected velocity field; and a network that learns the stream function, i.e. the scalar potential of the divergence free velocity vector field and which hence yields by construction divergence free corrections. Our experimental results show that the third approach based on the stream function outperforms the other two and not only improves the divergence freedom but in particular also the overall fidelity of the simulation.

show abstract

Section: Related Workmentioning

confidence: 99%

Structure Preservation for the Deep Neural Network Multigrid Solver

Margenberg,

Lessig,

Richter

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…These authors hence also use the deviation of the network prediction from the model as an additional penalty term in the training loss. Kasim et al [29] recently demonstrated neural network-based simulations for a broad range of applications, including fluid dynamics, by also optimizing the network architecture itself during train-ing process. Eichinger, Heinlein and Klawonn [33] use convolutional neural networks and techniques from image processing to learn flow patterns for the Navier-Stokes flow around objects of different shape.…”

Section: Related Workmentioning

confidence: 99%

A neural network multigrid solver for the Navier-Stokes equations

Margenberg,

Hartmann,

Lessig

et al. 2020

Preprint

View full text Add to dashboard Cite

We present a deep neural network multigrid solver (DNN-MG) that we develop for the instationary Navier-Stokes equations. DNN-MG improves computational efficiency using a judicious combination of a geometric multigrid solver and a recurrent neural network with memory. The multigrid method is used in DNN-MG to solve on coarse levels while the neural network corrects interpolated solutions on fine ones, thus avoiding the increasingly expensive computations that would have to be performed on there. A reduction in computation time is thereby achieved through DNN-MG's highly compact neural network. The compactness results from its design for local patches and the available coarse multigrid solutions that provides a "guide" for the corrections. A compact neural network with a small number of parameters also reduces training time and data. Furthermore, the network's locality facilitates generalizability and allows one to use DNN-MG trained on one mesh domain also on an entirely different one. We demonstrate the efficacy of DNN-MG for variations of the 2D laminar flow around an obstacle. For these, our method significantly improves the solutions as well as lift and drag functionals while requiring only about half the computation time of a full multigrid solution. We also show that DNN-MG trained for the configuration with one obstacle can be generalized to other time dependent problems that can be solved efficiently using a geometric multigrid method.

show abstract

“…Large initial-condition and perturbed parameter ensembles might contain hundreds of training points, while multi-model ensembles only contain tens. Using neural architecture search [46] or simpler models can help relieve this to some extent.…”

Section: (A) Challenges (I) Small Training Data Sizesmentioning

confidence: 99%

Reproducing Internal Variability with Few Ensemble Runs

Castruccio

Sanderson

et al. 2019

Journal of Climate

View full text Add to dashboard Cite

While large climate model ensembles are invaluable tools for physically consistent climate prediction, they also present a large burden in terms of computational resources and storage requirements. A complementary approach to large initial-condition ensembles is to train a stochastic generator on fewer runs. While simulations from a statistical model cannot capture the complexity of climate model runs, they can address some specific scientific questions of interest, such as sampling the variability of regional trends. We demonstrate this potential by comparing simulations from a large ensemble and a stochastic generator trained with only four runs, and show that the variability of regional temperature trends is almost indistinguishable. Training stochastic generators on fewer runs might prove especially useful in the context of large climate model intercomparison projects where creating large ensembles for each model is not possible.

show abstract

Building high accuracy emulators for scientific simulations with deep neural architecture search

Cited by 80 publications

References 38 publications

Structure Preservation for the Deep Neural Network Multigrid Solver

Structure Preservation for the Deep Neural Network Multigrid Solver

A neural network multigrid solver for the Navier-Stokes equations

Reproducing Internal Variability with Few Ensemble Runs

Contact Info

Product

Resources

About