Asynchronous Two-level Checkpointing Scheme for Large-scale Adjoints in the Spectral-Element Solver Nek5000

Schanen, Michel; Marin, Oana; Zhang, Hong; Anitescu, Mihai

doi:10.1016/j.procs.2016.05.444

Cited by 19 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonlinear optimal perturbations were calculated with nek5000 using a gradient-based optimisation algorithm that relies on integrating backward in time the adjoint equations to compute the gradient (Pringle & Kerswell 2010). A checkpointing and revolve strategy was adopted in order to limit the memory requirements (Schanen et al 2016). The initial condition is updated at each step of the optimization using the gradient rotation algorithm (Foures et al 2013).…”

Section: Turbulent Kinetic Energy Budgetmentioning

confidence: 99%

The vanishing of strong turbulent fronts in bent pipes

2019

View full text Add to dashboard Cite

Isolated patches of turbulence in transitional straight pipes are sustained by a strong instability at their upstream front, where the production of turbulent kinetic energy (TKE) is up to five times higher than in the core. Direct numerical simulations presented in this paper show no evidence of such strong fronts if the pipe is bent. We examine the temporal and spatial evolution of puffs and slugs in a toroidal pipe with pipe-to-torus diameter ratio δ = D/d = 0.01 at several subcritical Reynolds numbers. Results show that the upstream overshoot of TKE production is at most one-and-a-half times the value in the core and that the average cross-flow fluctuations at the front are up to three times lower if compared to a straight pipe, while attaining similar values in the core. Localised turbulence can be sustained at smaller energies through a redistribution of turbulent fluctuations and vortical structures by the in-plane Dean motion of the mean flow. This asymmetry determines a strong localisation of TKE production near the outer bend, where linear and nonlinear mechanisms optimally amplify perturbations. We further observe a substantial reduction of the range of Reynolds numbers for longlived intermittent turbulence, in agreement with experimental data from the literature. Moreover, no occurrence of nucleation of spots through splitting could be detected in the range of parameters considered. Based on the present results, we argue that this mechanism gradually becomes marginal as the curvature of the pipe increases and the transition scenario approaches a dynamical switch from subcritical to supercritical.

show abstract

Section: Turbulent Kinetic Energy Budgetmentioning

confidence: 99%

The vanishing of strong turbulent fronts in bent pipes

2019

View full text Add to dashboard Cite

show abstract

“…Therefore large memory or space (∼10 4 GB in the present case) is required when solving the adjoint. Note that this constraint on memory can be mitigated by implementing a checkpointing scheme at the cost of extra calls of equations (2.4) (Schanen et al 2016). Considering the sign of the viscous term and the time derivative term of this adjoint equation, it should be integrated backwards from t = T to t = 0.…”

Section: Optimal Inflow Perturbationmentioning

confidence: 99%

Far-wake meandering induced by atmospheric eddies in flow past a wind turbine

Mao

Sørensen

2018

J. Fluid Mech.

View full text Add to dashboard Cite

A novel algorithm is developed to calculate the nonlinear optimal boundary perturbations in three-dimensional incompressible flow. An optimal step length in the optimization loop is calculated without any additional calls to the Navier–Stokes equations. The algorithm is applied to compute the optimal inflow eddies for the flow around a wind turbine to clarify the mechanisms behind wake meandering, a phenomenon usually observed in wind farms. The turbine is modelled as an actuator disc using an immersed boundary method with the loading prescribed as a body force. At Reynolds number (based on free-stream velocity and turbine radius) $Re=1000$, the most energetic inflow perturbation has a frequency $\unicode[STIX]{x1D714}=0.8$–2, and is in the form of an azimuthal wave with wavenumber $m=1$ and the same radius as the actuator disc. The inflow perturbation is amplified by the strong shear downstream of the edge of the disc and then tilts the rolling-up vortex rings to induce wake meandering. This mechanism is verified by studying randomly perturbed flow at $Re\leqslant 8000$. At five turbine diameters downstream of the disc, the axial velocity oscillates at a magnitude of more than 60 % of the free-stream velocity when the magnitude of the inflow perturbation is 6 % of the free-stream wind speed. The dominant Strouhal number of the wake oscillation is 0.16 at $Re=3000$ and keeps approximately constant at higher $Re$. This Strouhal number agrees well with previous experimental findings. Overall the observations indicate that the well-observed stochastic wake meandering phenomenon appearing far downstream of wind turbines is induced by large-scale (the same order as the turbine rotor) and low-frequency free-stream eddies.

show abstract

“…Computed in reverse order of the primal computation, the solution at every time‐step must be recovered for each adjoint time‐step, requiring a careful combination of memory check‐pointing, disk check‐pointing, and recomputation. Nek5000 has recently incorporated a novel adjoint check‐pointing scheme that uses all system resources, memory, disks, and archive, proving that adjoint computations of nonlinear problems are feasible at large scale …”

Section: Contributions Algorithm and Implementationmentioning

confidence: 99%

Performance study of sustained petascale direct numerical simulation on Cray XC40 systems

Hadri

Parsani

Hutchinson

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary We present in this paper a comprehensive performance study of highly efficient extreme scale direct numerical simulations of secondary flows, using an optimized version of Nek5000. Our investigations are conducted on various Cray XC40 systems, using a very high‐order spectral element method. Single‐node efficiency is achieved by auto‐generated assembly implementations of small matrix multiplies and key vector‐vector operations, streaming lossless I/O compression, aggressive loop merging, and selective single precision evaluations. Comparative studies across different Cray XC40 systems at scale, Trinity (LANL), Cori (NERSC), and ShaheenII (KAUST) show that a Cray programming environment, network configuration, parallel file system, and burst buffer all have a major impact on the performance. All three systems possess a similar hardware with similar CPU nodes and parallel file system, but they have different theoretical peak network bandwidths, different OSs, and different versions of the programming environment. Our study reveals how these slight configuration differences can be critical in terms of performance of the application. We also find that with 9216 nodes (294 912 cores) on Trinity XC40 the applications sustain petascale performance, as well as 50% of peak memory bandwidth over the entire solver (500 TB/s in aggregate). On 3072 Xeon Phi nodes of Cori, we reach 378 TFLOP/s with an aggregated bandwidth of 310 TB/s, corresponding to time‐to‐solution 2.11× faster than obtained with the same number of (dual‐socket) Xeon nodes.

show abstract

Asynchronous Two-level Checkpointing Scheme for Large-scale Adjoints in the Spectral-Element Solver Nek5000

Cited by 19 publications

References 9 publications

The vanishing of strong turbulent fronts in bent pipes

The vanishing of strong turbulent fronts in bent pipes

Far-wake meandering induced by atmospheric eddies in flow past a wind turbine

Performance study of sustained petascale direct numerical simulation on Cray XC40 systems

Contact Info

Product

Resources

About