Designing a ROCm-Aware MPI Library for AMD GPUs: Early Experiences

Khorassani, Kawthar Shafie; Hashmi, Jahanzeb Maqbool; Chu, C. Y. Cyrus; Chen, Chen-Chun; Subramoni, Hari; Panda, Dhabaleswar K.

doi:10.1007/978-3-030-78713-4_7

Cited by 9 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, we reduce the number of inter-node communications and offload it to the intranode interconnect. Since the inter-node interconnects have significantly lower bandwidth than intra-node interconnects [5], this can improve communication performance.…”

Section: Algorithm 1: Intra-node Communication Load Matrixmentioning

confidence: 99%

“…The broad deployment of AMD GPUs in top supercomputers emphasizes the importance of optimizing AMD GPU communications. Improvements to Inter-GPU communication bandwidth have been found to correspond to improvements in application performance [5]. The multi-GPU computing nodes in AMD platforms are equipped with the Infinity Fabric TM link.…”

Section: Introductionmentioning

confidence: 99%

“…The multi-GPU computing nodes in AMD platforms are equipped with the Infinity Fabric TM link. These interconnects provide significantly improved bandwidth between devices compared to traditional PCIe interconnects [5], so it is desirable to utilize the Infinity Fabric TM efficiently to provide improved performance on AMD GPU-based systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ROCm-Aware Leader-based Designs for MPI Neighbourhood Collectives

Temuçin,

Gazimirsaeed,

Grant

et al. 2024

ISC High Performance 2024 Research Paper Proceedings (39th International Conference)

View full text Add to dashboard Cite

MPI neighborhood collectives were introduced in the MPI-3.0 standard to support sparse communication patterns used by many applications. Simultaneously, GPU-Aware MPI communication has become a prominent part of modern systems. With the rise of AMD GPUs and their incorporation into upcoming exascale systems like Frontier, it has become essential to optimize communication libraries for AMD platforms. In this paper, we take advantage of the hardware and networking features of AMD GPUs to design efficient and scalable neighborhood collective operations: allgather and allgatherv. We evaluate the performance of the proposed design for Random Sparse Graph and Moore neighborhood micro-benchmarks as well as an SpMM kernel. The results show that we obtain up to 7.03x speedup for the Random Sparse Graph micro-benchmark, up to 3.82x for the Moore neighborhood micro-benchmark, and up to 2.29x speedup for the SpMM kernel.

show abstract

Section: Algorithm 1: Intra-node Communication Load Matrixmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ROCm-Aware Leader-based Designs for MPI Neighbourhood Collectives

Temuçin,

Gazimirsaeed,

Grant

et al. 2024

ISC High Performance 2024 Research Paper Proceedings (39th International Conference)

View full text Add to dashboard Cite

show abstract

“…While NVIDIA's CUDA is a more established GPU programming framework, AMD's ROCm 3 represents a universal platform for GPU-accelerated computing. ROCm introduced new numerical formats to support common open-source machine learning libraries such as TensorFlow and PyTorch; it also provides the means for porting NVIDIA CUDA code into AMD hardware 4 . It is important to note that AMD not only is catching up to the ROCm platform in the GPU computing race, but also recently introduced the new flagship GPU architecture AMD Instinct MI200 Series 5 to compete with the latest NVIDIA Ampere A100 GPU architecture 6 .…”

mentioning

confidence: 99%