Exploiting Common Neighborhoods to Optimize MPI Neighborhood Collectives

Mirsadeghi, Seyed H.; Triff, Jesper Larsson; Balaji, Pavan; Afsahi, Ahmad

doi:10.1109/hipc.2017.00047

Cited by 14 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, we conduct experiments between traces with randomized placement (as has been the case thus far) and a "linear" placement where task i is placed on network endpoint i. This places neighboring tasks in neighboring network endpoints with no fragmentation and preserves all placement optimizations and communication locality that the application contains [30,65]. The purpose is to clearly show how bandwidth steering can reconstruct locality in an execution where locality has been lost due to unfavorable placement and fragmentation on the system.…”

Section: Link Utilizationmentioning

confidence: 99%

Bandwidth steering in HPC using silicon nanophotonics

Shen

Teh

Meng

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

As bytes-per-FLOP ratios continue to decline, communication is becoming a bottleneck for performance scaling. This paper describes bandwidth steering in HPC using emerging reconfigurable silicon photonic switches. We demonstrate that placing photonics in the lower layers of a hierarchical topology efficiently changes the connectivity and consequently allows operators to recover from system fragmentation that is otherwise hard to mitigate using common task placement strategies. Bandwidth steering enables efficient utilization of the higher layers of the topology and reduces cost with no performance penalties. In our simulations with a few thousand network endpoints, bandwidth steering reduces static power consumption per unit throughput by 36% and dynamic power consumption by 14% compared to a reference fat tree topology. Such improvements magnify as we taper the bandwidth of the upper network layer. In our hardware testbed, bandwidth steering improves total application execution time by 69%, unaffected by bandwidth tapering.

show abstract

Section: Link Utilizationmentioning

confidence: 99%

Bandwidth steering in HPC using silicon nanophotonics

Shen

Teh

Meng

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…Somewhat related to our work are the sparse neighborhood collectives [11][12][13]15] in MPI, with which one can define a restricted set of neighbor processes and perform collectives on them. In our methodology, the neighboring processes in distinct communication stages may or may not communicate with each other depending on what submessages they forward.…”

Section: Related Workmentioning

confidence: 99%

Regularizing irregularly sparse point-to-point communications

Selvitopi

Aykanat

2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

This work tackles the communication challenges posed by the latency-bound applications with irregular communication patterns, i.e., applications with high average and/or maximum message counts. We propose a novel algorithm for reorganizing a given set of irregular point-to-point messages with the objective of reducing total latency cost at the expense of increased volume. We organize processes into a virtual process topology inspired by the k-ary n-cube networks and regularize irregular messages by imposing regular communication pattern(s) onto them. Exploiting this process topology, we propose a flexible store-and-forward algorithm to control the trade-off between latency and volume. Our approach is able to reduce the communication time of sparse-matrix multiplication with latency-bound instances drastically: up to 22.6× for 16K processes on a 3D Torus network and up to 7.2× for 4K processes on a Dragonfly network, with its performance getting better with increasing number of processes.

show abstract

“…The standard communicators do not cover such an algorithm. One would need to implement it for a “graph communicator” as “neighbourhood collective communication.” The communicator creating routine would need to detect the multiple all‐to‐all communication pattern . This is in general computationally expensive.…”

Section: Implications For Communication Librariesmentioning

confidence: 99%

“…20 The communicator creating routine would need to detect the multiple all-to-all communication pattern. 21 This is in general computationally expensive. One solution could be the use of the MPI_Info object for the creation of the communicator in order to pass the details about the collectives to the library.…”

Section: Implications For Communication Librariesmentioning

confidence: 99%

Optimized all‐to‐all communication on multicore architectures applied to FFTs with pencil decomposition

Jocksch¹,

Kraushaar²,

Daverio

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary All‐to‐all communication is a basic functionality of parallel communication libraries such as the Message Passing Interface (MPI). Typically, there are multiple different underlying algorithms, which are chosen according to message size. We propose a communication algorithm, which exploits the fact that modern supercomputers combine shared memory parallelism and distributed memory parallelism. The application example of our algorithm is FFTs with pencil decomposition. Furthermore, we propose an extension of the MPI standard in order to accommodate this and other algorithms in an efficient way.

show abstract

Exploiting Common Neighborhoods to Optimize MPI Neighborhood Collectives

Cited by 14 publications

References 17 publications

Bandwidth steering in HPC using silicon nanophotonics

Bandwidth steering in HPC using silicon nanophotonics

Regularizing irregularly sparse point-to-point communications

Optimized all‐to‐all communication on multicore architectures applied to FFTs with pencil decomposition

Contact Info

Product

Resources

About