The effect of load imbalances on the performance of Monte Carlo algorithms in LWR analysis

Abstract

A model is developed to predict the impact of particle load imbalances on the performance of domain-decomposed Monte Carlo neutron transport algorithms. Expressions for upper bound performance "penalties" are derived in terms of simple machine characteristics, material characterizations and initial particle distributions. The hope is that these relations can be used to evaluate tradeoffs among different memory decomposition strategies in next generation Monte Carlo codes, and perhaps as a metric for triggering particle redistribution in production codes.

Monte Carlo domain decomposition for robust nuclear reactor analysis

“…As discussed in [24], load imbalances are normally expected to occur as domains get smaller and leakage fractions increase, incurring significant parallel inefficiencies when scaling the number of domains. Several algorithmic strategies exist to address this problem.…”

“…To verify that the present derivation of the model upperbound is equivalent to that presented in [24], the previouslyreported leakage fraction and particle source data tallied from full-scale OpenMC runs on the Monte Carlo Performance Benchmark problem ( [29]) were re-processed by the present Table 3 Directly-calculated load imbalance penalties predicted by the unapproximated model equations from previously-tallied data. The full case used all data (compare to the mesh in Figure 2), whereas the restricted case used data only from assembly regions and stages where particles were run in assembly regions (compare to the restricted mesh in Figure 5).…”

“…The ability to model and predict the extent of particle communication costs and load imbalances has been investigated in [23,24] for a range of machine and problem parameters with a simplified MC code. In these analyses it is demonstrated that these costs can be computed from the peaking factor and spatial statistics of the problem (i.e., per-domain leakage fractions), and that only modest penalties are predicted for a typical reactor geometry.…”

“…As described in [24], the time τ needed to complete a perfectly load-balanced domain-decomposed Monte Carlo run using this simple scheme can be modeled as a combination of latency, bandwidth, and particle tracking components: τ = τ lat enc y + τ bandwid th + τ t r acking (1) It is straightforward to write this in terms of the number of synchronization stages M , the average number of particles P i run in each domain at stage i, and the average fractionλ i of particles that leak out of each domain at stage i:…”

“…In [24] the authors took this a step further, expanding the sums in Equations 2 and 3 to write them in terms of the load imbalance distribution at the initial particle tracking stage. For example,…”