Radiative Heat Transfer Calculation on 16384 GPUs Using a Reverse Monte Carlo Ray Tracing Approach with Adaptive Mesh Refinement

Humphrey, Alan; Sunderland, Daniel; Harman, Todd; Berzins, Martin

doi:10.1109/ipdpsw.2016.93

Cited by 14 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This radiation calculation, in which the radiative-flux divergence at each cell of the discretized domain is calculated, can take up to 50% of the overall CPU time per timestep [11] using the discrete ordinates method (DOM) [6], one of the standard approaches to computing radiative heat transfer. Using a reverse Monte Carlo ray tracing approach combined with a novel use of Uintah's adaptive mesh refinement infrastructure, this calculation has been made to scale to 262K cores [11], and further adapted to run on up to 16K GPUs [12]. The spatial transport sweeps method discussed in Sect.…”

Section: Related Workmentioning

confidence: 99%

“…Reverse Monte Carlo Ray Tracing (RMCRT) [11] has been implemented on both CPUs and GPUs [11,12], and is a method for solving Eq. 2 by tracing radiation rays from one cell to the next, as described in detail in [11,12]. Reverse Monte Carlo (as opposed to forward) is desirable because rays are then independent of all other ray tracing processes, and are trivially parallel.…”

Section: Reverse Monte Carlo Ray Tracingmentioning

confidence: 99%

“…When computation resources are abundant and the solid-angle [11] can be well resolved, RMCRT is preferred to discrete-ordinates, where numerical diffusion hurts accuracy. This method has been made to scale up to 256K CPU cores and up to 16K GPUs within the Uintah framework [11,12,19], and is used for the large-scale, GPU-based boiler simulations, as detailed in [19].…”

Section: Reverse Monte Carlo Ray Tracingmentioning

confidence: 99%

See 2 more Smart Citations

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

Kumar

Humphrey

Usher

et al. 2018

Supercomputing Frontiers

Self Cite

View full text Add to dashboard Cite

The need to scale next-generation industrial engineering problems to the largest computational platforms presents unique challenges. This paper focuses on data management related problems faced by the Uintah simulation framework at a production scale of 260K processes. Uintah provides a highly scalable asynchronous many-task runtime system, which in this work is used for the modeling of a 1000 megawatt electric (MWe) ultra-supercritical (USC) coal boiler. At 260K processes, we faced both parallel I/O and visualization related challenges, e.g., the default file-per-process I/O approach of Uintah did not scale on Mira. In this paper we present a simple to implement, restructuring based parallel I/O technique. We impose a restructuring step that alters the distribution of data among processes. The goal is to distribute the dataset such that each process holds a larger chunk of data, which is then written to a file independently. This approach finds a middle ground between two of the most common parallel I/O schemes-file per process I/O and shared file I/O-in terms of both the total number of generated files, and the extent of communication involved during the data aggregation phase. To address scalability issues when visualizing the simulation data, we developed a lightweight renderer using OSPRay, which allows scientists to visualize the data interactively at high quality and make production movies. Finally, this work presents a highly efficient and scalable radiation model based on the sweeping method, which significantly outperforms previous approaches in Uintah, like discrete ordinates. The integrated approach allowed the USC boiler problem to run on 260K CPU cores on Mira.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Reverse Monte Carlo Ray Tracingmentioning

confidence: 99%

Section: Reverse Monte Carlo Ray Tracingmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

Kumar

Humphrey

Usher

et al. 2018

Supercomputing Frontiers

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, many Monte Carlo codes have been developed on graphical processing units for many diverse fields and applications such as finance [18] and molecular dynamics [19]. On the other hand, to the authors knowledge, the only MC method applied to the solution of thermal radiation implemented on GPU was developed by Humphrey et al [20] for grey gas applications. Their code showed excellent scaling capabilities up to 16834 GPUs, proving the feasibility of the GPU MC concept for thermal radiation.…”

Section: Introductionmentioning

confidence: 99%

A fast GPU Monte Carlo radiative heat transfer implementation for coupling with direct numerical simulation

Silvestri

Pečnik

2019

Journal of Computational Physics: X

View full text Add to dashboard Cite

We implemented a fast Reciprocal Monte Carlo algorithm, to accurately solve radiative heat transfer in turbulent flows of non-grey participating media that can be coupled to fully resolved turbulent flows, namely to Direct Numerical Simulation (DNS). The spectrally varying absorption coefficient is treated in a narrow-band fashion with a correlated-k distribution. The implementation is verified with analytical solutions and validated with results from literature and line-by-line Monte Carlo computations. The method is implemented on GPU with a thorough attention to memory transfer and computational efficiency. The bottlenecks that dominate the computational expenses are addressed and several techniques are proposed to optimize the GPU execution. By implementing the proposed algorithmic accelerations, a speed-up of up to 3 orders of magnitude can be achieved, while maintaining the same accuracy.

show abstract

“…Adaptive mesh refinement (AMR) is a key ingredient in many application domains [6,8,10,12,15,16,22]. It allows to resolve "areas of interest", such as wave fronts, shocks or (moving) boundaries with a fine mesh while allowing a Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Section: Introductionmentioning

confidence: 99%

Asagi

Rettenberger

Meister

Bäder

et al. 2016

Proceedings of the Exascale Applications and Software Conference 2016

View full text Add to dashboard Cite

We present ASAGI, an open-source library with a simple interface to access Cartesian material and geographic datasets in massively parallel simulations with dynamically adaptive mesh refinement (AMR). ASAGI distributes geographic datasets over all compute nodes storing only a portion of the dataset on each node. An automatic replication mechanism copies the data between nodes to assure fast local access even after load migration in the application. We demonstrate ASAGI's preparedness for up-to-petascale simulations in three use cases. We simulate a Tsunami on 512 cores and a porous media flow on up to 8,192 cores of SuperMUC with the AMR framework sam(oa) 2. We also run an earthquake simulation with SeiSol on 65,536 cores. For all applications, ASAGI provides large complex 3D material datasets required for the realistic scenarios. The NUMA-awareness of ASAGI turned out to be especially useful for the hybrid MPI+OpenMP parallelization of both codes. CCS Concepts •Computing methodologies → Massively parallel and high-performance simulations; •Applied computing → Earth and atmospheric sciences; •Theory of computation → Massively parallel algorithms;

show abstract

Radiative Heat Transfer Calculation on 16384 GPUs Using a Reverse Monte Carlo Ray Tracing Approach with Adaptive Mesh Refinement

Cited by 14 publications

References 31 publications

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

A fast GPU Monte Carlo radiative heat transfer implementation for coupling with direct numerical simulation

Asagi

Contact Info

Product

Resources

About