Efficient Congestion Management for High-Speed Interconnects using Adaptive Routing

The development of new exascale supercomputers has dramatically increased the need for fast, high-performance networking technology. Efficient network topologies, such as Dragonfly+, have been introduced to meet the demands of data-intensive applications and to match the massive computing power of GPUs and accelerators. However, these supercomputers still face performance variability mainly caused by the network that affects system and application performance. This study comprehensively analyzes performance variability on a large-scale HPC system with Dragonfly+ network topology, focusing on factors such as communication patterns, message size, job placement locality, MPI collective algorithms, and overall system workload. The study also proposes an easy-to-measure metric for estimating network background traffic generated by other users, which can be used to estimate the performance of our job accurately. The insights gained from this study contribute to improving performance predictability, enhancing job placement policies and MPI algorithm selection, and optimizing resource management strategies in supercomputers.

show abstract

Efficient Congestion Management for High-Speed Interconnects using Adaptive Routing

Cited by 4 publications

References 27 publications

An Analysis of Long-Tailed Network Latency Distribution and Background Traffic on Dragonfly+

An Analysis of Long-Tailed Network Latency Distribution and Background Traffic on Dragonfly+

Congestion management in high-performance interconnection networks using adaptive routing notifications

Analysis and prediction of performance variability in large-scale computing systems

Contact Info

Product

Resources

About