Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning

Abstract

There is a growing interest in developing data-driven subgrid-scale (SGS) models for largeeddy simulation (LES) using machine learning (ML). In a priori (offline) tests, some recent studies have found ML-based data-driven SGS models that are trained on high-fidelity data (e.g., from direct numerical simulation, DNS) to outperform baseline physics-based models and accurately capture the inter-scale transfers, both forward (diffusion) and backscatter. While promising, instabilities in a posteriori (online) tests and inabilities to generalize to a different flow (e.g., with a higher Reynolds number, Re) remain as major obstacles in broadening the applications of such data-driven SGS models. For example, many of the same aforementioned studies have found instabilities that required often ad-hoc remedies to stabilize the LES at the expense of reducing accuracy. Here, using 2D decaying turbulence as the testbed, we show that deep fully convolutional neural networks (CNNs) can accurately predict the SGS forcing terms and the inter-scale transfers in a priori tests, and if trained with enough samples, lead to stable and accurate a posteriori LES-CNN. Further analysis attributes these instabilities to the disproportionally lower accuracy of the CNNs in capturing backscattering when the training set is small. We also show that transfer learning, which involves re-training the CNN with a small amount of data (e.g., 1%) from the new flow, enables accurate and stable a posteriori LES-CNN for flows with 16× higher Re (as well as higher grid resolution if needed). These results show the promise of CNNs with transfer learning to provide stable, accurate, and generalizable LES for practical use.

Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES

“…To derive the equations for LES, we apply Gaussian filtering [2,3,20], denoted by (•), to Eqs. (1a)-(1b) to obtain…”

“…Functional models, such as the Smagorinsky model [1] and its dynamic variants [16][17][18], are often developed by considering the inter-scale interactions (e.g., energy transfers). While producing low c (< 0.6) between the true and predicted SGS terms in a priori analysis [19][20][21][22], these functional models usually provide numerically stable a posteriori LES, at least partly due to their dissipative nature. Thus, developing SGS models that perform well in both a priori and a posteriori analyses has remained a long-lasting research focus.…”

“…For example, Maulik et al [25,39] and Xie et al [40][41][42] have, respectively, developed local data-driven closures for 2D decaying homogeneous isotropic turbulence (2D-DHIT) and 3D incompressible and compressible turbulence using multilayer perceptron artificial neural networks (ANNs); also see [22,[43][44][45][46]. Zanna and Bolton [29,47], Beck and colleagues [26,48], Pawar et al [19], Guan et al [20], and Subel et al [49] developed non-local closures, e.g., using convolutional neural networks (CNNs), for ocean circulation, 3D-DHIT, 2D-DHIT, and forced 1D Burgers' turbulence, respectively. While finding outstanding results in a priori analyses, in many cases, these studies also reported instabilities in a posteriori analyses, requiring further modifications to the learnt closures for stabilization.…”

“…More recently, Guan et al [20] showed that increasing the size of the training set alone can lead to stable and accurate a posteriori LES (as well as high c in a priori analysis) even with physicsagnostic CNNs 1 . This was attributed to the following: Big training datasets obtained from filtered DNS (FDNS) snapshots can provide sufficient information such that the physical constraints and processes (e.g., backscattering) are correctly learnt by data-driven methods, leading to a stable and accurate LES in both a priori and a posteriori analyses.…”

“…This was attributed to the following: Big training datasets obtained from filtered DNS (FDNS) snapshots can provide sufficient information such that the physical constraints and processes (e.g., backscattering) are correctly learnt by data-driven methods, leading to a stable and accurate LES in both a priori and a posteriori analyses. However, in the small-data regime, the physical constraints and processes may not be captured correctly, and the inaccuracy of the datadrivenly predicted SGS term (particularly inaccuracies in backscattering) can result in unstable or unphysical LES [20]. Note that as discussed later in Section 3, whether a training dataset is big or small depends on both the number of samples and the inter-sample correlations; thus it depends on the total length of the available DNS dataset, which can be limited due to computational constraints.…”