This paper develops the Non-Intrusive Least Squares Shadowing (NILSS) method, which computes the sensitivity for long-time averaged objectives in chaotic dynamical systems. In NILSS, we represent a tangent solution by a linear combination of one inhomogeneous tangent solution and several homogeneous tangent solutions. Next, we solve a least squares problem using this representation; thus, the resulting solution can be used for computing sensitivities. NILSS is easy to implement with existing solvers. In addition, for chaotic systems with many degrees of freedom but few unstable modes, NILSS has a low computational cost. NILSS is applied to two chaotic PDE systems: the Lorenz 63 system and a CFD simulation of flow over a backward-facing step. In both cases, the sensitivities computed by NILSS reflect the trends in the long-time averaged objectives of dynamical systems.where f (u, s) : R m × R → R m is a smooth function, u is the state, and s is the parameter. The initial condition u 0 is a smooth function of φ. A solution u(t) is called the primal solution.In this paper, The objective is a long-time averaged quantity. To define it, we first let J(u, s) : R m × R → R be a continuous function that represents the instantaneous objective function. The objective is obtained by averaging J over a infinitely long trajectory:J T depends on s, φ, and T , while J ∞ is determined only by s and u 0 . Here we make the assumption of ergodicity [3], which means that u 0 , hence φ does not affect J ∞ . As a result, J ∞ only depends on s. The purpose of this paper is to develop an algorithm that computes the sensitivity d J ∞ /ds. The sensitivity can help scientists and engineers design products [4,5], control processes and systems [6,7], solve inverse problems [8], estimate simulation errors [9,10,11], assimilate measurement data [12,13] and quantify uncertainties [14]. When a dynamical system is chaotic, computing a meaningful d J ∞ /ds is challenging, since in general:That is, if we fix u 0 (φ), the process of T → ∞ does not commute with differentiation with respect to s [15]. As a result, the transient method, which employs the conventional tangent method with a fixed u 0 , does not converge to the correct sensitivity for chaotic systems. In fact, the transient method diverges most of the time [15]. Many sensitivity analysis methods have been developed to compute d J ∞ /ds. The conventional methods include the finite difference and transient method. The ensemble method, developed by Lea et al. [16,17], computes the sensitivity by averaging results from the transient method over an ensemble of trajectories. Another recent approach is based on the fluctuation dissipation theorem (FDT), as seen in [18,19,20,21,22,23].In this research study, we consider the Least Squares Shadowing (LSS) approach, developed by Wang, Hu and Blonigan [24,14]. LSS computes a bounded shift of a trajectory under an infinitesimal parameter change, which is called the LSS solution. The LSS solution can then be used to compute the derivative d J ∞ /ds. LSS...

This paper uses compressible flow simulation to analyze the hyperbolicity, shadowing directions, and sensitivities of a weakly turbulent three dimensional cylinder flow at Reynolds number 525 and Mach number 0.1.By computing the first 40 Covariant Lyapunov Vectors (CLVs), we find that unstable CLVs are active in the near-wake region, whereas stable CLVs are active in the far-wake region. This phenomenon is related to hyperbolicity since it shows that CLVs point to different directions; it also suggests that for open flows there is a large fraction of CLVs that are stable. However, due to the extra neutral CLV and the occasional tangencies between CLVs, our system is not uniform hyperbolic.By the Non-intrusive least-squares shadowing (NILSS) algorithm, we compute shadowing directions and sensitivities of long-time-averaged objectives. Our results suggest that shadowing methods may be valid for general chaotic fluid problems.

No abstract

For a parameterized hyperbolic system du dt = f (u, s) the derivative of the ergodic average J = lim T →∞ 1 T T 0 J(u(t), s) to the parameter s can be computed via the Least Squares Shadowing algorithm (LSS). We assume that the sytem is ergodic which means that J depends only on s (not on the initial condition of the hyperbolic system). The algorithm solves a constrained least squares problem and, from the solution to this problem, computes the desired derivative d J ds . The purpose of this paper is to prove that the value given by the LSS algorithm approaches the exact derivative when the timespan used to formulate the least squares problem grows to infinity. It then illustrates the convergence result through a numerical example.The differential equation is assumed to be uniformly hyperbolic (details in section 3). We are also given a C 1 cost function J(u, s) : U × R → R and assume that the system is ergodic, i.e., the infinite time average :depends on s but does not depend on the initial condition u(0). The differentiability of J with respect to s has been proven by Ruelle [1]. Obtaining an estimation of d J ds is crucial in many computational and engineering problems. Indeed, many applications involve simulations of nonlinear dynamical systems that exhibit a chaotic behavior. For instance, chaos can be encountered in the following fields : climate and weather prediction [2], turbulent combustion simulation [3], nuclear reactor physics [4], plasma dynamics in fusion [5] and multi-body problems [6]. The quantities of interest are often time averages or expected values of some cost function J. Estimating the derivative of J is particularly valuable in:• Numerical optimization. The derivative of J with respect to a design parameter s is used by gradient-based algorithms in order to efficiently optimize the design parameters in high dimensional design spaces (see [7]). • Uncertainty quantification. The derivative of J with respect to a parameter s gives a useful information for assessing the error and uncertainty in the computed J (see [8]).

We develop the NILSAS algorithm, which performs adjoint sensitivity analysis of chaotic systems via computing the adjoint shadowing direction. NILSAS constrains its minimization to the adjoint unstable subspace, and can be implemented with little modification to existing adjoint solvers. The computational cost of NILSAS is independent of the number of parameters. We demonstrate NILSAS on the Lorenz 63 system and a weakly turbulent three-dimensional flow over a cylinder.

For hyperbolic diffeomorphisms, we define adjoint shadowing directions as a bounded inhomogeneous adjoint solution whose initial condition has zero component in the unstable adjoint direction. For hyperbolic flows, we define adjoint shadowing directions similarly, with the additional requirement that the average of its inner-product with the trajectory direction is zero. In both cases, we show unique existence of adjoint shadowing directions, and how they can be used for adjoint sensitivity analysis. Our work set a theoretical foundation for efficient adjoint sensitivity methods for longtime-averaged objectives such as NILSAS.

We present the Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS) algorithm for computing sensitivities of long-time averaged quantities in chaotic dynamical systems. FD-NILSS does not require tangent solvers, and can be implemented with little modification to existing numerical simulation software. We also give a formula for solving the least-squares problem in FD-NILSS, which can be applied in NILSS as well. Finally, we apply FD-NILSS for sensitivity analysis of a chaotic flow over a 3-D cylinder at Reynolds number 525, where FD-NILSS computes accurate sensitivities and the computational cost is in the same order as the numerical simulation.

We devise a new algorithm, called the linear response algorithm, for differentiating SRB measures with respect to system parameters, where SRB measures are fractal limiting stationary measures of chaotic systems. The algorithm is illustrated on an example which is difficult for previous algorithms.The algorithm works for chaos on general manifolds with any unstable dimension, u. The algorithm is efficient and robust: its main cost is solving u many first-order and second-order tangent equations, and it does not compute oblique projections. The convergence to the true derivative is proved for uniform hyperbolic systems.The core of our algorithm is the first numerical treatment of the unstable divergence, a central object in the linear response theory for fractal attractors. We give a new characterization that the unstable divergence can be expressed by the solution of a renormalized second-order tangent equation, whose second derivative is taken in a modified shadowing direction, computed by the non-intrusive shadowing algorithm.