High Performance MPI over the Slingshot Interconnect: Early Experiences

Khorassani, Kawthar Shafie; Chen, Chen Chun; Ramesh, Bharath; Shafi, Aamir; Subramoni, Hari; Panda, Dhabaleswar K.

doi:10.1145/3491418.3530773

Cited by 7 publications

(4 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 6 shows the Infinity Fabric TM topology of the GPUs in Cluster B. The nodes are connected with HPE's Slingshot Interconnect [29] and runs the GNU/Linux distribution SUSE Linux, kernel version 5.3.18.…”

Section: A Experimental Setupmentioning

confidence: 99%

ROCm-Aware Leader-based Designs for MPI Neighbourhood Collectives

Temuçin,

Gazimirsaeed,

Grant

et al. 2024

ISC High Performance 2024 Research Paper Proceedings (39th International Conference)

View full text Add to dashboard Cite

MPI neighborhood collectives were introduced in the MPI-3.0 standard to support sparse communication patterns used by many applications. Simultaneously, GPU-Aware MPI communication has become a prominent part of modern systems. With the rise of AMD GPUs and their incorporation into upcoming exascale systems like Frontier, it has become essential to optimize communication libraries for AMD platforms. In this paper, we take advantage of the hardware and networking features of AMD GPUs to design efficient and scalable neighborhood collective operations: allgather and allgatherv. We evaluate the performance of the proposed design for Random Sparse Graph and Moore neighborhood micro-benchmarks as well as an SpMM kernel. The results show that we obtain up to 7.03x speedup for the Random Sparse Graph micro-benchmark, up to 3.82x for the Moore neighborhood micro-benchmark, and up to 2.29x speedup for the SpMM kernel.

show abstract

Section: A Experimental Setupmentioning

confidence: 99%

ROCm-Aware Leader-based Designs for MPI Neighbourhood Collectives

Temuçin,

Gazimirsaeed,

Grant

et al. 2024

ISC High Performance 2024 Research Paper Proceedings (39th International Conference)

View full text Add to dashboard Cite

show abstract

“…The HPE-Cray EX system and software management stacks support IaC capabilities through key innovative technologies (Cray System Management). These systems are characterised by two distinguishing features that are relevant for our study:• Software-defined, Ethernet-compliant, high-performance networking stack – interoperability with the data centre networks has been among the key designed objectives of the Cray-HPE Slingshot interconnect technology, in addition to support for the HPC, MPI payloads (Khorassani et al, 2022). As a result, it is compatible with Ethernet while offering additional essential features for the low-latency and high-bandwidth needs of HPC applications such as adaptive routing and congestion control.…”

Section: Cray-hpe Shasta Ex System and Software Stackmentioning

confidence: 99%

“…• Software-defined, Ethernet-compliant, high-performance networking stack – interoperability with the data centre networks has been among the key designed objectives of the Cray-HPE Slingshot interconnect technology, in addition to support for the HPC, MPI payloads (Khorassani et al, 2022). As a result, it is compatible with Ethernet while offering additional essential features for the low-latency and high-bandwidth needs of HPC applications such as adaptive routing and congestion control.…”

Section: Cray-hpe Shasta Ex System and Software Stackmentioning

confidence: 99%

“…These are nevertheless not a replacement for a shared supercomputing ecosystem where a single job can potentially scale to the entire low-latency, high-performance networking domain, which can be defined as an RDMA (Remote Direct Memory Access) network scaling in contrast with typical L2 and L3 cloud network scaling (Alam et al, 2020). All compute resources of Alps supercomputer are connected via a low-latency, high-bandwidth proprietary interconnect technology called SlingShot (Khorassani et al, 2022). With the vCluster solution, we obtain native performance and scaling characteristics for large-scale MPI jobs, across the whole infrastructure and within individual tenants.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Versatile software-defined HPC and cloud clusters on Alps supercomputer for diverse workflows

Alam

Gila

Klein

et al. 2023

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Supercomputers have been driving innovations for performance and scaling benefiting several scientific applications for the past few decades. Yet their ecosystems remain virtually unchanged when it comes to integrating distributed data-driven workflows, primarily due to rather rigid access methods and restricted configuration management options. X-as-a-Service model of cloud has introduced, among other features, a developer-centric DevOps approach empowering developers of infrastructure, platform to software artefacts, which, unfortunately contemporary supercomputers still lack. We introduce vClusters (versatile software-defined clusters), which is based on Infrastructure-as-code (IaC) technology. vClusters approach is a unique fusion of HPC and cloud technologies resulting in a software-defined, multi-tenant cluster on a supercomputing ecosystem, that, together with software-defined storage, enable DevOps for complex, data-driven workflows like grid middleware, alongside a classic HPC platform. IaC has been a commonplace in cloud computing, however, it lacked adoption within multi-Petascale ecosystems due to concerns related to performance and interoperability with classic HPC data centres’ ecosystems. We present an overview of the Swiss National Supercomputing Centre’s flagship Alps ecosystem as an implementation target for vClusters for HPC and data-driven workflows. Alps is based on the Cray-HPE Shasta EX supercomputing platform that includes an IaC compliant, microservices architecture (MSA) management system, which we leverage for demonstrating vClusters usage for our diverse operational workflows. We provide implementation details of two operational vClusters platforms: a classic HPC platform that is used predominantly by hundreds of users running thousands of large-scale numerical simulations batch jobs; and a widely used, data-intensive, Grid computing middleware platform used for CERN Worldwide LHC Computing Grid (WLCG) operations. The resulting solution showcases reuse and reduction of common configuration recipes across vCluster implementations, minimising operational change management overheads while introducing flexibility for managing artefacts for DevOps required by diverse workflows.

show abstract