Large Scale Parallel Solution of Incompressible Flow Problems Using Uintah and Hypre

Schmidt, John A.; Berzins, Martin; Thornock, Jeremy; Saad, Tony; Sutherland, James C.

doi:10.1109/ccgrid.2013.10

Cited by 27 publications

(25 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In reacting flow simulations, the main computational cost is the solution of the large number of systems of linear equations required by the Discrete Ordinates Method. While the solution of these systems can be made to scale [28], it is important to reduce this cost. With this cost reduction in mind, more recent work has been based upon the use of more efficient Reverse Monte Carlo Ray Tracing (RMCRT) methods, e.g.…”

Section: The Arches Combustion Simulation Componentmentioning

confidence: 99%

“…The Uintah task-based model lends itself better to the use of Pthreads rather than OpenMP. This approach has led to the development of a multi-threaded MPI runtime system, including a threaded task scheduler that has enabled Uintah to show excellent strong and weak scaling up to 196K cores on the DOE Jaguar XT5 system and good initial scaling to 262k cores on the upgraded DOE Jaguar XK6 system [28]. Using this approach has reduced Uintah's on-node memory usage by up to 80% [21].…”

Section: Introductionmentioning

confidence: 97%

See 1 more Smart Citation

Radiation modeling using the Uintah heterogeneous CPU/GPU runtime system

Humphrey

Meng

Berzins

et al. 2012

Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging From the eXtreme to Th

View full text Add to dashboard Cite

The Uintah Computational Framework was developed to provide an environment for solving fluid-structure interaction problems on structured adaptive grids on large-scale, long-running, data-intensive problems. Uintah uses a combination of fluid-flow solvers and particle-based methods for solids, together with a novel asynchronous task-based approach with fully automated load balancing. Uintah demonstrates excellent weak and strong scalability at full machine capacity on XSEDE resources such as Ranger and Kraken, and through the use of a hybrid memory approach based on a combination of MPI and Pthreads, Uintah now runs on up to 262k cores on the DOE Jaguar system. In order to extend Uintah to heterogeneous systems, with ever-increasing CPU core counts and additional onnode GPUs, a new dynamic CPU-GPU task scheduler is designed and evaluated in this study. This new scheduler enables Uintah to fully exploit these architectures with support for asynchronous, outof-order scheduling of both CPU and GPU computational tasks. A new runtime system has also been implemented with an added multi-stage queuing architecture for efficient scheduling of CPU and GPU tasks. This new runtime system automatically handles the details of asynchronous memory copies to and from the GPU and introduces a novel method of pre-fetching and preparing GPU memory prior to GPU task execution. In this study this new design is examined in the context of a developing, hierarchical GPUbased ray tracing radiation transport model that provides Uintah with additional capabilities for heat transfer and electromagnetic wave propagation. The capabilities of this new scheduler design are tested by running at large scale on the modern heterogeneous systems, Keeneland and TitanDev, with up to 360 and 960 GPUs respectively. On these systems, we demonstrate significant speedups per GPU against a standard CPU core for our radiation problem.

show abstract

Section: The Arches Combustion Simulation Componentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

Radiation modeling using the Uintah heterogeneous CPU/GPU runtime system

Humphrey

Meng

Berzins

et al. 2012

Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging From the eXtreme to Th

View full text Add to dashboard Cite

show abstract

“…Good scalability has been achieved with the Taylor Green Vortex Problem described in [51]. This involved a careful use of both data structures and options in the Hypre code so it is far from clear that the same scalability will transfer from Titan to the other two machines.…”

Section: Archesmentioning

confidence: 96%

“…• Turbulent combustion on a fixed-mesh requiring large-scale linear solves with the Hypre iterative solver [51] again used on the DOE Titan system. These three applications have very different communications requirements and work patterns.…”

Section: Introductionmentioning

confidence: 99%

Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers

Meng

Humphrey

Schmidt

et al. 2013

Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Present trends in high performance computing present formidable challenges for applications code using multicore nodes possibly with accelerators and/or co-processors and reduced memory while still attaining scalability. Software frameworks that execute machineindependent applications code using a runtime system that shields users from architectural complexities offer a possible solution. The Uintah framework for example, solves a broad class of large-scale problems on structured adaptive grids using fluid-flow solvers coupled with particle-based solids methods. Uintah executes directed acyclic graphs of computational tasks with a scalable asynchronous and dynamic runtime system for CPU cores and/or accelerators/coprocessors on a node. Uintah's clear separation between application and runtime code has led to scalability increases of 1000x without significant changes to application code. This methodology is tested on three leading Top500 machines; OLCF Titan, TACC Stampede and ALCF Mira using three diverse and challenging applications problems. This investigation of scalability with regard to the different processors and communications performance leads to the overall conclusion that the adaptive DAG-based approach provides a very powerful abstraction for solving challenging multiscale multi-physics engineering problems on some of the largest and most powerful computers available today.

show abstract

“…The GIMP method was integrated into the publically available Uintah package [9] but can only be used for conventional sequential or parallel computing on CPUs [15]. The GPU parallel computing strategy developed in this study can be combined with any MPM using explicit integration schemes, although its operational procedure will be described in terms of the GIMP method.…”

Section: Gpu Parallelisation Of Mpmmentioning

confidence: 99%