The adjoint method, among other sensitivity analysis methods, can fail in chaotic dynamical systems. The result from these methods can be too large, often by orders of magnitude, when the result is the derivative of a long time averaged quantity. This failure is known to be caused by ill-conditioned initial value problems. This paper overcomes this failure by replacing the initial value problem with the well-conditioned "least squares shadowing (LSS) problem". The LSS problem is then linearized in our sensitivity analysis algorithm, which computes a derivative that converges to the derivative of the infinitely long time average. We demonstrate our algorithm in several dynamical systems exhibiting both periodic and chaotic oscillations.

Computational methods for sensitivity analysis are invaluable tools for aerodynamics research and engineering design. However, traditional sensitivity analysis methods break down when applied to long-time averaged quantities in turbulent fluid flow fields, specifically those obtained using high-fidelity turbulence simulations. This is because of a number of dynamical properties of turbulent and chaotic fluid flows, most importantly high sensitivity of the initial value problem, popularly known as the "butterfly effect".The recently developed least squares shadowing (LSS) method avoids the issues encountered by traditional sensitivity analysis methods by approximating the "shadow trajectory" in phase space, avoiding the high sensitivity of the initial value problem. The following paper discusses how the least squares problem associated with LSS is solved. Two methods are presented and are demonstrated on a simulation of homogeneous isotropic turbulence and the Kuramoto-Sivashinsky (KS) equation, a 4th order chaotic partial differential equation. We find that while LSS computes fairly accurate gradients, faster, more efficient linear solvers are needed to apply both LSS methods presented in this paper to larger simulations.

Computational methods for sensitivity analysis are invaluable tools for scientists and engineers investigating a wide range of physical phenomena. However, many of these methods fail when applied to chaotic systems, such as the Kuramoto-Sivashinsky (K-S) equation, which models a number of different chaotic systems found in nature. The following paper discusses the application of a new sensitivity analysis method developed by the authors to a modified K-S equation. We find that least squares shadowing sensitivity analysis computes accurate gradients for solutions corresponding to a wide range of system parameters.

Sensitivity analysis methods are important tools for research and design with simulations. Many important simulations exhibit chaotic dynamics, including scale-resolving turbulent fluid flow simulations. Unfortunately, conventional sensitivity analysis methods are unable to compute useful gradient information for long-time-averaged quantities in chaotic dynamical systems. Sensitivity analysis with least squares shadowing (LSS) can compute useful gradient information for a number of chaotic systems, including simulations of chaotic vortex shedding and homogeneous isotropic turbulence. However, this gradient information comes at a very high computational cost. This paper presents multiple shooting shadowing (MSS), a more computationally efficient shadowing approach than the original LSS approach. Through an analysis of the convergence rate of MSS, it is shown that MSS can have lower memory usage and run time than LSS.

No abstract

We present a reformulation of unsteady turbulent flow simulations. The initial condition is relaxed and information is allowed to propagate both forward and backward in time. Simulations of chaotic dynamical systems with this reformulation can be proven to be well-conditioned time domain boundary value problems. The reformulation can enable scalable parallel-in-time simulation of turbulent flows. I. NEED FOR SPACE-TIME PARALLELISMThe use of computational fluid dynamics (CFD) in science and engineering can be categorized into Analysis and Design. A CFD Analysis performs a simulation on a set of manually picked parameter values. The flow field is then inspected to gain understanding of the flow physics. Scientific and engineering decisions are then made based on understanding of the flow field. Analysis based on high fidelity turbulent flow simulations, particular Large Eddy Simulations, is a rapidly growing practice in complex engineering applications 12 .CFD based Design goes beyond just performing individual simulations, towards sensitivity analysis, optimization, control, uncertainty quantification and data based inference. Design is enabled by Analysis capabilities, but often requires more rapid turnaround. For example, an engineer designer or an optimization software needs to perform a series of simulations, modifying the geometry based on previous simulation results. Each simulation must complete within at most a few hours in an industrial design environment. Most current practices of design use steady state CFD solvers, employing RANS (Reynolds Averaged NavierStokes) models for turbulent flows. Design using high fidelity, unsteady turbulent flow simulations has been investigated in academia 3 . Despite their great potential, high fidelity design is infeasible in an industrial setting because each simulation typically takes days to weeks. FIG. 1:Exponential increase of high performance computing power, primarily sustained by increased parallelism in the past decade. Data originate from top500.org. GFLOPS, TFLOPS, PFLOPS and EFLOPS represent 10 9 , 10 12 , 10 15 and 10 18 FLoating point Operations Per Second, respectively.The inability of performing high fidelity turbulent flow simulations in short turnaround time is a barrier to the game-changing technology of high fidelity CFD-based design. Nevertheless, development in High Performance Computing (HPC), as shown in Figure 1, promises to delivery in about ten years computing hardware a thousand times faster than those available today. This will be achieved through extreme scale a) Corresponding

Gradient-based sensitivity analysis has proven to be an enabling technology for many applications, including design of aerospace vehicles. However, conventional sensitivity analysis methods break down when applied to long-time averages of chaotic systems. This breakdown is a serious limitation because many aerospace applications involve physical phenomena that exhibit chaotic dynamics, most notably high-resolution large-eddy and direct numerical simulations of turbulent aerodynamic flows. A recently proposed methodology, Least Squares Shadowing (LSS), avoids this breakdown and advances the state of the art in sensitivity analysis for chaotic flows. The first application of LSS to a chaotic flow simulated with a large-scale computational fluid dynamics solver is presented. The LSS sensitivity computed for this chaotic flow is verified and shown to be accurate, but the computational cost of the current LSS implementation is high.

For a parameterized hyperbolic system du dt = f (u, s) the derivative of the ergodic average J = lim T →∞ 1 T T 0 J(u(t), s) to the parameter s can be computed via the Least Squares Shadowing algorithm (LSS). We assume that the sytem is ergodic which means that J depends only on s (not on the initial condition of the hyperbolic system). The algorithm solves a constrained least squares problem and, from the solution to this problem, computes the desired derivative d J ds . The purpose of this paper is to prove that the value given by the LSS algorithm approaches the exact derivative when the timespan used to formulate the least squares problem grows to infinity. It then illustrates the convergence result through a numerical example.The differential equation is assumed to be uniformly hyperbolic (details in section 3). We are also given a C 1 cost function J(u, s) : U × R → R and assume that the system is ergodic, i.e., the infinite time average :depends on s but does not depend on the initial condition u(0). The differentiability of J with respect to s has been proven by Ruelle [1]. Obtaining an estimation of d J ds is crucial in many computational and engineering problems. Indeed, many applications involve simulations of nonlinear dynamical systems that exhibit a chaotic behavior. For instance, chaos can be encountered in the following fields : climate and weather prediction [2], turbulent combustion simulation [3], nuclear reactor physics [4], plasma dynamics in fusion [5] and multi-body problems [6]. The quantities of interest are often time averages or expected values of some cost function J. Estimating the derivative of J is particularly valuable in:• Numerical optimization. The derivative of J with respect to a design parameter s is used by gradient-based algorithms in order to efficiently optimize the design parameters in high dimensional design spaces (see [7]). • Uncertainty quantification. The derivative of J with respect to a parameter s gives a useful information for assessing the error and uncertainty in the computed J (see [8]).