Scalable MPI Collectives using SHARP: Large Scale Performance Evaluation on the TACC Frontera System

Ramesh, Bharath; Suresh, Kaushik Kandadi; Sarkauskas, Nick; Bayatpour, Mohammadreza; Hashmi, Jahanzeb Maqbool; Subramoni, Hari; Panda, Dhabaleswar K.

doi:10.1109/exampi52011.2020.00007

Cited by 3 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of nodes was varied from 2 to 128 with one MPI process per node. The data for MVAPICH2‐X‐SHArP is collected from Reference 49 running on TACC Frontera HPC system. The figure demonstrates that the FPGA‐based approach may be able to outperform high‐end CPU clusters optimized with SHArP technology.…”

Section: Discussionmentioning

confidence: 99%

“…Scaling comparison of the Allreduce operation between MVAPICH2‐X‐SHArP 49 and MPI‐FPGA with radix‐28 switches for 2048 bytes up to 128 nodes…”

Section: Discussionmentioning

confidence: 99%

“…A hardware‐based streaming aggregation capability was later added to provide high bandwidth for large messages 48 . Finally, the authors in Reference 49 designed, implemented, and evaluated SHArP‐based solutions for MPI_Reduce and MPI_Barrier in MVAPICH2‐X.…”

Section: Related Workmentioning

confidence: 99%

“…As a further evaluation we compared MPI-FPGA for an indirect network based on radix-28 switches with a SHArP-based MVAPICH2-X implementation. 49 Figure 13 depicts the scaling of MPI-FPGA with radix-28 switches (FPGA-indirect-28) and the SHArP-based solution in the MVA-PICH implementation (MVAPICH2-X-SHArP) for Allreduce with 2048 byte messages. The number of nodes was varied from 2 to 128 with one MPI process per node.…”

Section: Indirect Network Studymentioning

confidence: 99%

See 3 more Smart Citations

Reconfigurable switches for high performance and flexible MPI collectives

Haghi

Guo

Xiong

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

There has been much effort in offloading MPI collective operations into hardware.But while NIC-based collective acceleration is well-studied, offloading their processing into the switching fabric, despite numerous advantages, has been much more limited.A major problem with fixed logic implementations is that either only a fraction of the possible collective communication is accelerated or that logic is wasted in the applications that do not need a particular capability. Using reconfigurable logic has numerous advantages: exactly the required operations can be implemented; the level of desired performance can be specified; and new, possibly complex, operations can be defined and implemented. We have designed an in-switch collective accelerator, MPI-FPGA, and demonstrated its use with seven MPI collectives and over a set of benchmarks and proxy applications (MiniApps). The accelerator uses a novel two-level switch design containing fully pipelined vectorized aggregation logic units. Essential to this work is providing support for sub-communicator collectives that enables communicators of arbitrary shape, and that is scalable to large systems. A streaming interface improves the performance for long messages. While this reconfigurable design is generally applicable, we prototype it with an FPGA-centric cluster. A sample MPI-FPGA design in a direct network achieves considerable speedups over conventional clusters in the most likely scenarios. We also present results for indirect networks with reconfigurable high-radix switches and show that this approach is competitive with SHArP technology for the subset of operations that SHArP supports. MPI-FPGA is fully integrated into MPICH and is transparent to MPI applications.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Scaling comparison of the Allreduce operation between MVAPICH2‐X‐SHArP 49 and MPI‐FPGA with radix‐28 switches for 2048 bytes up to 128 nodes…”

Section: Discussionmentioning

confidence: 99%