Effect of Parallel Computing Environment on the Solution Consistency of CFD Simulations—Focused on IC Engines

Abstract

For CFD results to be useful in IC engine analysis, simulation results should be accurate and consistent. However, with wide spread use of parallel computing nowadays, it has been reported that a model would not give the same results against the same input when the parallel computing environment is changed. The effect of parallel environment on simulation results needs to be carefully investigated and understood. In this paper, the solution inconsistency of parallel CFD simulations is investigated. First, the concept of solution inconsistency on parallel computing is reviewed, followed by a systematic CFD simulations specific to IC engine applications. The solution inconsistency against the number of CPU cores was examined using a commercial CFD code CONVERGE. A test matrix was specifically designed to examine the core number effect on engine flow, spray and combustion submodels performance. It was found that the flow field simulation during the gas exchange process is the most sensitive to the number of cores among all submodels examined. An engineering solution was developed where local upwind scheme was used to control the variability, which showed good performance. The implication of the observed inconsistency was also discussed.

Race conditions and data partitioning: risks posed by common errors to reproducible parallel simulations

“…The difficulties of programming parallel simulations are concretely demonstrated by reproducibility challenges that have come to light in prominent simulation programs from several domains of science and engineering, including computational fluid dynamics, traffic simulations, nuclear engineering, 14–20 and in various parallel simulation tools. 14,15,21…”

“…Discussions of these errors, particularly in the context of new parallel programmers, are abundant. 14–20…”

“…The difficulties of programming parallel simulations are concretely demonstrated by reproducibility challenges that have come to light in prominent simulation programs from several domains of science and engineering, including computational fluid dynamics, traffic simulations, nuclear engineering, [14][15][16][17][18][19][20] and in various parallel simulation tools. 14,15,21 While systematic studies of reproducibility are sometimes published for widely used simulation tools, such as those cited here, similar studies for models built and used by individuals or small teams are difficult to find. Nonetheless, if widely used simulations and simulation tools can exhibit unexpected non-reproducibility, it is reasonable to assume that these issues exist in other, less scrutinized parallel simulations.…”

“…Discussions of these errors, particularly in the context of new parallel programmers, are abundant. [14][15][16][17][18][19][20] A race condition is an error that causes the ordering of actions to be a consequence of how threads and processes are scheduled rather than the logic of the simulation model. If left unresolved, race conditions in a simulation program can make it infeasible to both reproduce a single simulation execution and as well as recreate statistical distributions constructed from numerous simulation executions.…”

Using scheduling entropy amplification in CUDA/OpenMP code to exhibit non-reproducibility issues

“…Furthermore they can come with a significant performance penalty. This is a major issue as nondeterminism of floating-point calculations in parallel programs causes validation and debugging issues, and may even lead to deadlocks [1], [2], [3], [4].…”