Scalability of OpenFOAM with Large Eddy Simulations and DNS on High-Performance Systems

Axtmann, Gabriel; Rist, Ulrich

doi:10.1007/978-3-319-47066-5_28

Cited by 13 publications

(12 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can observe that the rho-CentralRK4Foam solver takes less time to communicate, which lead to the improved parallel performance found in previous tests. In addition, among the MPI subroutines that are relevant for communication (i.e., called every time step in the simulation), MPI_waitall() was the one that employed most of the time (see Figure 5), which is in line with previous profiling (see, for example, Axtmann and Rist [24]).…”

Section: Strong Scaling Testssupporting

confidence: 70%

Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme

Paoli

D‘Mello

2020

Scientific Programming

View full text Add to dashboard Cite

Compressible density-based solvers are widely used in OpenFOAM, and the parallel scalability of these solvers is crucial for large-scale simulations. In this paper, we report our experiences with the scalability of OpenFOAM’s native rhoCentralFoam solver, and by making a small number of modifications to it, we show the degree to which the scalability of the solver can be improved. The main modification made is to replace the first-order accurate Euler scheme in rhoCentralFoam with a third-order accurate, four-stage Runge-Kutta or RK4 scheme for the time integration. The scaling test we used is the transonic flow over the ONERA M6 wing. This is a common validation test for compressible flows solvers in aerospace and other engineering applications. Numerical experiments show that our modified solver, referred to as rhoCentralRK4Foam, for the same spatial discretization, achieves as much as a 123.2% improvement in scalability over the rhoCentralFoam solver. As expected, the better time resolution of the Runge–Kutta scheme makes it more suitable for unsteady problems such as the Taylor–Green vortex decay where the new solver showed a 50% decrease in the overall time-to-solution compared to rhoCentralFoam to get to the final solution with the same numerical accuracy. Finally, the improved scalability can be traced to the improvement of the computation to communication ratio obtained by substituting the RK4 scheme in place of the Euler scheme. All numerical tests were conducted on a Cray XC40 parallel system, Theta, at Argonne National Laboratory.

show abstract

Section: Strong Scaling Testssupporting

confidence: 70%

Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme

Paoli

D‘Mello

2020

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…Also the parallel scalability of the codes influences the total computational costs. OpenFOAM, for example, is characterized by quite modest scalability, compared to the other considered codes [29,30]. In our simulations with OpenFOAM, we used a sufficiently low number of cores, such that we operate only in the range of good parallel efficiency.…”

Section: Number Of Gridpointsmentioning

confidence: 99%

Comparison of computational codes for direct numerical simulations of turbulent Rayleigh–Bénard convection

et al. 2018

View full text Add to dashboard Cite

Computational codes for direct numerical simulations of Rayleigh-Bénard (RB) convection are compared in terms of computational cost and quality of the solution. As a benchmark case, RB convection at Ra = 10 8 and Pr = 1 in a periodic domain, in cubic and cylindrical containers is considered. A dedicated second-order finite-difference code (AFID/RBflow) and a specialized fourth-order finite-volume code (Goldfish) are compared with a general purpose finite-volume approach (OpenFOAM) and a general purpose spectral-element code (Nek5000). Reassuringly, all codes provide predictions of the average heat transfer that converge to the same values. The computational costs, however, are found to differ considerably. The specialized codes AFID/RBflow and Goldfish are found to excel in efficiency, outperforming the general purpose flow solvers Nek5000 and OpenFOAM by an order of magnitude with an error on the Nusselt number Nu below 5%. However, we find that Nu alone is not sufficient to assess the quality of the numerical results: in fact, instantaneous snapshots of the temperature field from a near wall region obtained for deliberately under-resolved simulations using Nek5000 clearly indicate inadequate flow resolution even when Nu is converged. Overall, dedicated special purpose codes for RB convection are found to be more efficient than general purpose codes.

show abstract

“…The reasons for the different behavior can be manifold and range from the influence of the load balancing method to cache effects and communication effects. A detailed discussion of these influencing factors can be found in [72,76,77].…”

Section: Simulation Performancementioning

confidence: 99%

“…The estimation of parallel 579 scalability requires extensive testing due to the strong influence by the cells per core ratio, the load 580 balancing method and the connection between the nodes of the cluster system. Comprehensive studies 581 that deal with the parallel scalability of OpenFOAM and OpenLB can be found in [43,72,73].…”

mentioning

confidence: 99%

Evaluation of a Near-Wall-Modeled Large Eddy Lattice Boltzmann Method for the Analysis of Complex Flows Relevant to IC Engines

et al. 2020

View full text Add to dashboard Cite

In this paper, we compare the capabilities of two open source near-wall-modeled large eddy simulation (NWM-LES) approaches regarding prediction accuracy, computational costs and ease of use to predict complex turbulent flows relevant to internal combustion (IC) engines. The applied open source tools are the commonly used OpenFOAM, based on the finite volume method (FVM), and OpenLB, an implementation of the lattice Boltzmann method (LBM). The near-wall region is modeled by the Musker equation coupled to a van Driest damped Smagorinsky-Lilly sub-grid scale model to decrease the required mesh resolution. The results of both frameworks are compared to a stationary engine flow bench experiment by means of particle image velocimetry (PIV). The validation covers a detailed error analysis using time-averaged and root mean square (RMS) velocity fields. Grid studies are performed to examine the performance of the two solvers. In addition, the differences in the processes of grid generation are highlighted. The performance results show that the OpenLB approach is on average 32 times faster than the OpenFOAM implementation for the tested configurations. This indicates the potential of LBM for the simulation of IC engine-relevant complex turbulent flows using NWM-LES with computationally economic costs.

show abstract

Scalability of OpenFOAM with Large Eddy Simulations and DNS on High-Performance Systems

Cited by 13 publications

References 5 publications

Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme

Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme

Comparison of computational codes for direct numerical simulations of turbulent Rayleigh–Bénard convection

Evaluation of a Near-Wall-Modeled Large Eddy Lattice Boltzmann Method for the Analysis of Complex Flows Relevant to IC Engines

Contact Info

Product

Resources

About