Supporting efficient execution in heterogeneous distributed computing environments with cactus and globus

Abstract

Improvements in the performance of processors and networks make it both feasible and interesting to treat collections of workstations, servers, clusters, and supercomputers as integrated computational resources, or Grids. However, the highly heterogeneous and dynamic nature of such Grids can make application development difficult. Here we describe an architecture and prototype implementation for a Grid-enabled computational framework based on Cactus, the MPICH-G2 Grid-enabled message-passing library, and a variety of specialized features to support efficient execution in Grid environments. We have used this framework to perform record-setting computations in numerical relativity, running across four supercomputers and achieving scaling of 88% (1140 CPU's) and 63% (1500 CPUs). The problem size we were able to compute was about five times larger than any other previous run. Further, we introduce and demonstrate adaptive methods that automatically adjust computational parameters during run time, to increase dramatically the efficiency of a distributed Grid simulation, without modification of the application and without any knowledge of the underlying network connecting the distributed computers.

The Cactus Framework and Toolkit: Design and Applications

“…There are plans to add features to allow the processor topology to be optimised for node-based machines such as the IBM SP or Hitachi SR8000 which consist of small (typically 8 or 16 processor) nodes connected together to produce a larger machine. The load-balancing algorithm in PUGH is also customisable, and a modified version of PUGH has been used in large scale distributed computing [19].…”

“…Its flexible and portable architecture make it a good match for the varying needs of Grid applications. In particular, it has traditionally been used for many remote and distributed computing demonstrations [37,38,39,19]. Because of the abstractions available in Cactus, it has been possible to modify the various data distribution, message passing, and IO layers so they function efficiently in a distributed environment, without modifying the applications themselves [19].…”

“…In particular, it has traditionally been used for many remote and distributed computing demonstrations [37,38,39,19]. Because of the abstractions available in Cactus, it has been possible to modify the various data distribution, message passing, and IO layers so they function efficiently in a distributed environment, without modifying the applications themselves [19]. For example, it has been possible to take very complex, production code for solving Einstein's equations which is coupled to GR hydrodynamics to simulate colliding neutron stars, and distributed across multiple supercomputers on different continents, while it is remote visualised and controlled from yet another location.…”

Cactus Application: Performance Predictions in Grid Environments

“…The experimental setup, including details on the middleware infrastructure used (Globus and MPICH-G2), is presented in [3]. The first experiment used 2 supercomputers with up to 64 processors each.…”

“…The first experiment used 2 supercomputers with up to 64 processors each. The second, large-scale experiment involved 4 supercomputers of a total of 1500 processors (described in [2]). In both cases the results measured were at most 15% less than our predictions.…”