We propose a modeling paradigm, termed field inversion and machine learning (FIML), that seeks to comprehensively harness data from sources such as high-fidelity simulations and experiments to aid the creation of improved closure models for computational physics applications. In contrast to inferring model parameters, this work uses inverse modeling to obtain corrective, spatially distributed functional terms, offering a route to directly address model-form errors. Once the inference has been performed over a number of problems that are representative of the deficient physics in the closure model, machine learning techniques are used to reconstruct the model corrections in terms of variables that appear in the closure model. These reconstructed functional forms are then used to augment the closure model in a predictive computational setting. As a first demonstrative example, a scalar ordinary differential equation is considered, wherein the model equation has missing and deficient terms. Following this, the methodology is extended to the prediction of turbulent channel flow. In both of these applications, the approach is demonstrated to be able to successfully reconstruct functional corrections and yield accurate predictive solutions while providing a measure of model form uncertainties.

Reduced models of nonlinear dynamical systems require closure, or the modelling of the unresolved modes. The Mori-Zwanzig procedure can be used to derive formally closed evolution equations for the resolved physics. In these equations, the unclosed terms are recast as a memory integral involving the time history of the resolved variables. While this procedure does not reduce the complexity of the original system, these equations can serve as a mathematically consistent basis to develop closures based on memory approximations. In this scenario, knowledge of the memory kernel is paramount in assessing the validity of a memory approximation. Unravelling the memory kernel requires solving the orthogonal dynamics, which is a high-dimensional partial differential equation that is intractable, in general. A method to estimate the memory kernel , using full-order solution snapshots, is proposed. The key idea is to solve a pseudo orthogonal dynamics equation, which has a convenient Liouville form, instead. This ersatz arises from the assumption that the semi-group of the orthogonal dynamics is a composition operator for one observable. The method is exact for linear systems. Numerical results on the Burgers and Kuramoto-Sivashinsky equations demonstrate that the proposed technique can provide valuable information about the memory kernel.

This work uses the Mori-Zwanzig (M-Z) formalism, a concept originating from non-equilibrium statistical mechanics, as a basis for the development of coarse-grained models of turbulence. The mechanics of the generalized Langevin equation (GLE) are considered and insight gained from the orthogonal dynamics equation is used as a starting point for model development. A class of sub-grid models is considered which represent non-local behavior via a finite memory approximation (Stinis, P., "Mori-Zwanzig reduced models for uncertainty quantification I: Parametric uncertainty," arXiv:1211.4285, 2012.), the length of which is determined using a heuristic that is related to the spectral radius of the Jacobian of the resolved variables. The resulting models are intimately tied to the underlying numerical resolution and are capable of approximating non-Markovian effects. Numerical experiments on the Burgers equation demonstrate that the M-Z-based models can accurately predict the temporal evolution of the total kinetic energy and the total dissipation rate at varying mesh resolutions. The trajectory of each resolved mode in phase-space is accurately predicted for cases where the coarse-graining is moderate. LES of homogeneous isotropic turbulence and the Taylor Green Vortex show that the M-Z-based models are able to provide excellent predictions, accurately capturing the sub-grid contribution to energy transfer. Lastly, LES of fully developed channel flow demonstrate the applicability of M-Z-based models to non-decaying problems. It is notable that the form of the closure is not imposed by the modeler, but is rather derived from the mathematics of the coarse-graining, highlighting the potential of M-Z-based techniques to define LES closures.

The development of reduced models for complex multiscale problems remains one of the principal challenges in computational physics. The optimal prediction framework of Chorin et al. [1], which is a reformulation of the Mori-Zwanzig (M-Z) formalism of non-equilibrium statistical mechanics, provides a methodology for the development of mathematically-derived reduced models of dynamical systems. Several promising models have emerged from the optimal prediction community and have found application in molecular dynamics and turbulent flows. In this work, a new M-Z-based closure model that addresses some of the deficiencies of existing methods is developed. The model is constructed by exploiting similarities between two levels of coarse-graining via the Germano identity of fluid mechanics and by assuming that memory effects have a finite temporal support. The appeal of the proposed model, which will be referred to as the 'dynamic-MZ-τ' model, is that it is parameter-free and has a structural form imposed by the mathematics of the coarse-graining process (rather than the phenomenological assumptions made by the modeler, such as in classical subgrid scale models). To promote the applicability of M-Z models in general, two procedures are presented to compute the resulting model form, helping to bypass the tedious error-prone algebra that has proven to be a hindrance to the construction of M-Z-based models for complex dynamical systems. While the new formulation is applicable to the solution of general partial differential equations, demonstrations are presented in the context of Large Eddy Simulation closures for Burgers equation, decaying homogeneous turbulence, and turbulent channel flow. The performance of the model and validity of the underlying assumptions are investigated in detail.

We formulate a new projection-based reduced-ordered modeling technique for non-linear dynamical systems. The proposed technique, which we refer to as the Adjoint Petrov-Galerkin (APG) method, is derived by decomposing the generalized coordinates of a dynamical system into a resolved coarse-scale set and an unresolved fine-scale set. A Markovian finite memory assumption within the Mori-Zwanzig formalism is then used to develop a reduced-order representation of the coarse-scales. This procedure leads to a closed reduced-order model that displays commonalities with the adjoint stabilization method used in finite elements. The formulation is shown to be equivalent to a Petrov-Galerkin method with a non-linear, time-varying test basis, thus sharing some similarities with the Least-Squares Petrov-Galerkin method. Theoretical analysis examining a priori error bounds and computational cost is presented. Numerical experiments on the compressible Navier-Stokes equations demonstrate that the proposed method can lead to improvements in numerical accuracy, robustness, and computational efficiency over the Galerkin method on problems of practical interest. Improvements in numerical accuracy and computational efficiency over the Least-Squares Petrov-Galerkin method are observed in most cases.(G ROM) has been used successfully in a variety of problems. When applied to general non-self-adjoint and non-linear problems, however, theoretical analysis and numerical experiments have shown that Galerkin ROM lacks a priori guarantees of stability, accuracy, and convergence [9]. This last issue is particularly challenging as it demonstrates that enriching a ROM basis does not necessarily improve the solution [10]. The development of stable and accurate reduced-order modeling techniques for complex non-linear systems is the motivation for the current work.A significant body of research aimed at producing accurate and stable ROMs for complex non-linear problems exists in the literature. These efforts include, but are not limited to, "energy-based" inner products [9,11], symmetry transformations [12], basis adaptation [13,14], L 1 -norm minimization [15], projection subspace rotations [16], and least-squares residual minimization approaches [17,18,19,20,21,22,23,24]. The Least-Squares Petrov-Galerkin (LSPG) [22] method comprises a particularly popular leastsquares residual minimization approach and has been proven to be an effective tool for non-linear model reduction. Defined at the fully-discrete level (i.e., after spatial and temporal discretization), LSPG relies on least-squares minimization of the FOM residual at each time-step. While the method lacks a priori stability guarantees for general non-linear systems, it has been shown to be effective for complex problems of interest [24,23,25]. Additionally, as it is formulated as a minimization problem, physical constraints such as conservation can be naturally incorporated into the ROM formulation [26]. At the fully-discrete level, LSPG is sensitive to both the time integration scheme as ...

Traditional sub-grid models for Large Eddy Simulations (LES) of turbulence rely on theoretical and physical arguments and are known to be inadequate in many applications. This work investigates the use of the Mori-Zwanzig formalism, a concept that originates from non-equilibrium statistical mechanics, to develop sub-grid models for LES. The mechanics of the generalized Langevin equation (GLE) are considered and a methodology for directly solving the orthogonal dynamics equation is presented. This process is shown to provide insight into the form of the GLE and the effects of coarse-graining. This knowledge is used to derive a class of sub-grid models that are capable of consistently representing non-local memory effects. The models are applied to the viscous Burgers equation and homogeneous isotropic turbulence. The models are shown to be in good agreement with high resolution simulations and outperform Smagorinsky-type models.

This work proposes a machine-learning framework for modeling the error incurred by approximate solutions to parameterized dynamical systems. In particular, we extend the machine-learning error models (MLEM) framework proposed in Ref.[15] to dynamical systems. The proposed Time-Series Machine-Learning Error Modeling (T-MLEM) method constructs a regression model that maps features-which comprise error indicators that are derived from standard a posteriori error-quantification techniques-to a random variable for the approximate-solution error at each time instance. The proposed framework considers a wide range of candidate features, regression methods, and additive noise models. We consider primarily recursive regression techniques developed for time-series modeling, including both classical time-series models (e.g., autoregressive models) and recurrent neural networks (RNNs), but also analyze standard non-recursive regression techniques (e.g., feed-forward neural networks) for comparative purposes. Numerical experiments conducted on multiple benchmark problems illustrate that the long short-term memory (LSTM) neural network, which is a type of RNN, outperforms other methods and yields substantial improvements in error predictions over traditional approaches.

Reynolds Averaged Navier-Stokes (RANS) turbulence models have historically been developed using a combination of theoretical/physical/mathematical arguments, modeler expertise, and empirical data-fitting. As a consequence of the loss of information incurred during the ensemble averaging process, RANS model development has always been data-driven, to an extent, out of necessity. With the profusion of high resolution simulations and experimental methods, there exists a prodigious opportunity to more comprehensively inform closure models with data and improve their utlility in practical predictions of complex turbulent flows. In contrast to inferring model parameters, this work uses inverse modeling to directly obtain corrective, spatially distributed terms that are consistent with the predictive RANS models, thus offering a route to address the issue of model discrepancy which has long plagued turbulence model development. The posterior distributions of the inferred terms are quantified using Bayesian inversion and sampling. Results are presented for turbulent channel flow and flows with wall curvature. While the source terms inferred in this work are problem dependent, an ongoing subset of our work is to use machine learning methodologies to obtain general functional forms of the type that are inferred in this work. This paper specifically focuses on the mechanics of Bayesian inversion and its implications on the quantification of errors and uncertainties in turbulence modeling.