Adjoint sensitivity analysis on chaotic dynamical systems by Non-Intrusive Least Squares Adjoint Shadowing (NILSAS)

Abstract

We develop the NILSAS algorithm, which performs adjoint sensitivity analysis of chaotic systems via computing the adjoint shadowing direction. NILSAS constrains its minimization to the adjoint unstable subspace, and can be implemented with little modification to existing adjoint solvers. The computational cost of NILSAS is independent of the number of parameters. We demonstrate NILSAS on the Lorenz 63 system and a weakly turbulent three-dimensional flow over a cylinder.

Hyperbolicity, shadowing directions and sensitivity analysis of a turbulent three-dimensional flow

“…In other words, we can perform, at each time step, one matrix-matrix product, where the second matrix is composed of several tangent solutions, rather than performing several matrixvector products, where we need to reload the matrix for each product. † Additionally, as discussed in (Ni & Wang 2017;Ni et al 2019), FD-NILSS and NILSS have smaller marginal cost for new parameters; for cases with more parameters than objectives or cases when we have only adjoint solvers, the non-intrusive least-squares adjoint shadowing (NILSAS) method (Ni & Talnikar 2019a) can compute the gradient of one objective with respect to many parameters in one run.…”

Feasibility Analysis of Ensemble Sensitivity Computation in Turbulent Flows

“…To compute sensitivities with respect to design parameters, of statistically stationary quantities in chaotic systems, ensemble methods appear to be an appealing solution; they are both conceptually simpler and easier to implement than fluctuation-dissipation-based [21] and shadowing-based [20] For this, the most optimistic assumptions are made on the bias and variance associated with the ensemble sensitivity (ES) estimator, under the mathematical simplification of uniform hyperbolicity. We show that, with the integration time, in the best case, the bias decays exponentially at a problem-dependent rate and the variance increases exponentially at the rate of twice the largest Lyapunov exponent.…”

“…In this work, we present an analysis of the mean squared error of the ES method as a function of the computational cost for a certain class of systems called uniformly hyperbolic systems [19]. It is worth noting that at the time of writing of this paper, alternatives to the ES method [20,21] are under active investigation. Non-intrusive least squares shadowing (NILSS) [13,22] and its adjoint-variant [20,22] are methods that are conceptually based on the shadowing property of uniformly hyperbolic systems.…”

“…It is worth noting that at the time of writing of this paper, alternatives to the ES method [20,21] are under active investigation. Non-intrusive least squares shadowing (NILSS) [13,22] and its adjoint-variant [20,22] are methods that are conceptually based on the shadowing property of uniformly hyperbolic systems. This property enables the computation of a particular tangent solution that remains bounded in a long time window and sensitivities are estimated using this tangent solution.…”

Sensitivity analysis on chaotic dynamical systems by Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS)

“…For problems with a large number of unstable Lyapunov exponents, we suggest switching from FD-NILSS to NILSS or NILSAS, where we can take advantage of the vectorization of linear solvers and further accelerate computing homogeneous tangent or adjoint solutions, as discussed in [34]. Still, the cost of non-intrusive shadowing methods can get larger when m us is larger and the system becomes more chaotic.…”