Uintah is a software framework that provides an environment for solving fluid-structure interaction problems on structured adaptive grids for large-scale science and engineering problems involving the solution of partial differential equations. Uintah uses a combination of fluid flow solvers and particle-based methods for solids, together with adaptive meshing and a novel asynchronous task-based approach with fully automated load balancing. When applying Uintah to fluid-structure interaction problems, the combination of adaptive meshing and the movement of structures through space present a formidable challenge in terms of achieving scalability on large-scale parallel computers. The Uintah approach to the growth of the number of core counts per socket together with the prospect of less memory per core is to adopt a model that uses MPI to communicate between nodes and a shared memory model on-node so as to achieve scalability on large-scale systems. For this approach to be successful, it is necessary to design data structures that large numbers of cores can simultaneously access without contention. This scalability challenge is addressed here for Uintah, by the development of new hybrid runtime and scheduling algorithms combined with novel lock-free data structures, making it possible for Uintah to achieve excellent scalability for a challenging fluid-structure problem with mesh refinement on as many as 260K cores. ‡ Titan is a parallel computer under construction at Oak Ridge National Laboratory with about 299K CPU cores now with a large number of attached GPUs to be added in 2012. § Stampede is a parallel computer under construction at Texas Advanced Computing Center with two petaflops of CPU performance and eight petaflops of accelerator performance.
1389This work will start to address some of these challenges by developing scheduling algorithms and associated software that will be used to tackle fluid-structure interaction algorithms in which mesh refinement is used to resolve the structure. The vehicle for this work is the Uintah Computational Framework [2][3][4]. The broad range and challenging nature of Uintah applications and the software itself are described in [5]. The Uintah code was designed to solve fluid-structure interaction problems arising from a number of physical scenarios. Uintah uses a combination of computational fluid dynamics (CFD) algorithms in its implicit continuous-fluid Eulerian (ICE) solver [6,7] and couples ICE to a particle-based solver for solids known as the material point method (MPM) [8,9]. The adaptive mesh refinement (AMR) approach used in Uintah is that of multiple levels of regularly refined mesh patches. This mesh is also used as a scratch pad for the MPM calculation of the movement and deformation of the solid [10,11]. Uintah has been shown to scale successfully with many fluid and solid problems with adaptive meshes [12,13]; however, the scalability of fluid-structure interaction problems has proven to be somewhat more challenging. This is at least partly because the part...
