MPI Collective Communications on The Blue Gene/P Supercomputer: Algorithms and Optimizations

Faraj, Ahmad; Kumar, Sameer; Smith, Brian E.; Mamidala, Amith R.; Gunnels, John A.

doi:10.1109/hoti.2009.12

Cited by 36 publications

(33 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We do this by comparing theoretical bounds based on first principles gathered from the documentation [7] with benchmarked performance on MPI COMM WORLD on full allocations. Models for collective communications on other communicators and non-cubic allocations could be derived with a similar method.…”

Section: Process-to-node Mappingmentioning

confidence: 99%

“…This is the most accurate technique in our hierarchy and most convenient for the application developer. For example, the cost of a barrier on a BlueGene/P (BG/P) is T BAR = 0.95µs [7], independent of P .…”

Section: Previous Work and A General Approach To Modelingmentioning

confidence: 99%

“…The bandwidth of the collective network is 824 MiB/s [7]. IBM, however, did not release latency numbers for this network; thus, we resort to numerical methods for deriving a model.…”

Section: Process-to-node Mappingmentioning

confidence: 99%

See 2 more Smart Citations

Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues

Hoefler

Gropp

Thakur

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Designing and tuning parallel applications with MPI, particularly at large scale, requires understanding the performance implications of different choices of algorithms and implementation options. Which algorithm is better depends in part on the performance of the different possible communication approaches, which in turn can depend on both the system hardware and the MPI implementation. In the absence of detailed performance models for different MPI implementations, application developers often must select methods and tune codes without the means to realistically estimate the achievable performance and rationally defend their choices. In this paper, we advocate the construction of more useful performance models that take into account limitations on network-injection rates and effective bisection bandwidth. Since collective communication plays a crucial role in enabling scalability, we also provide analytical models for scalability of collective communication algorithms, such as broadcast, allreduce, and all-to-all. We apply these models to an IBM Blue Gene/P system and compare the analytical performance estimates with experimentally measured values. MotivationPerformance modeling of parallel applications leads to an understanding of their running time on parallel systems. To develop a model for an existing application or algorithm, one typically constructs a dependency graph of computations and communications from the start of the algorithm (input) to the end of the algorithm (output). This application model can then be matched to a machine model in order to estimate the run time of the algorithm on a particular architecture.Performance models can be used to make important early decisions about algorithmic choices. For example, to compute a three-dimensional Fast Fourier Transformation (3d FFT), one can either use a one-dimensional decomposition where each process computes full planes (2d FFTs) or a two-dimensional decomposition where each process computes sets of pencils (1d FFTs). If we assume

show abstract

Section: Process-to-node Mappingmentioning

confidence: 99%

Section: Previous Work and A General Approach To Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues

Hoefler

Gropp

Thakur

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…So, on a torus network, a message must travel l · d/2 hops at each step, rather than 1 hop as in SD-Cannon . The Blue Gene/P machine provides efficient broadcast collectives that work at a fine granularity and incur little latency overhead [6]. However, on a machine without this type of topology-aware collectives, SD-Cannon would have a strong advantage, as messages would need to travel fewer hops.…”

Section: Discussionmentioning

confidence: 99%

“…Each update requires a broadcast along a row or column of processors. If a higher-dimensional torus is flattened into each row and column of the mapping, rectangular collective algorithms [13,6,10] can utilize all dimensions of the network. Rectangular algorithms subdivide and pipeline the messages into edge-disjoint spanning trees formed by traversing the network in different dimensional orders.…”

Section: Introductionmentioning

confidence: 99%

Matrix Multiplication on Multidimensional Torus Networks

Solomonik

Demmel

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Blocked matrix multiplication algorithms such as Cannon's algorithm and SUMMA have a 2-dimensional communication structure. We introduce a generalized 'Split-Dimensional' version of Cannon's algorithm (SD-Cannon) with higher-dimensional and bidirectional communication structure. This algorithm is useful for higher-dimensional torus interconnects that can achieve more injection bandwidth than single-link bandwidth. On a bidirectional torus network of dimension d, SD-Cannon can lower the algorithmic bandwidth cost by a factor of up to d. With rectangular collectives, SUMMA also achieves the lower bandwidth cost but has a higher latency cost. We use Charm++ virtualization to efficiently map SD-Cannon on unbalanced and odd-dimensional torus network partitions. Our performance study on Blue Gene/P demonstrates that an MPI version of SD-Cannon can exploit multiple communication links and improve performance.

show abstract

Retraction

2011

Concurrency and Computation

View full text Add to dashboard Cite

MPI Collective Communications on The Blue Gene/P Supercomputer: Algorithms and Optimizations

Cited by 36 publications

References 7 publications

Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues

Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues

Matrix Multiplication on Multidimensional Torus Networks

Retraction

Contact Info

Product

Resources

About