Early evaluation of direct large-scale InfiniBand networks with adaptive routing

Daryin, Alexander N.; Korzh, Anton

doi:10.14529/jsfi140303

Cited by 3 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mellanox has also implemented a patented solution to handle out-of-order delivery packets [3]. Through these and several additional development steps, the AR offered by InfiniBand is reported to have become more powerful and useful, but evaluation is limited to some experiences from the early versions of AR [6] the pre-exascale system CORAL [29] that only focuses on bisection bandwidth without considering fairness.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Routing in InfiniBand Hardware

Rocher-González

Gran

Reinemo

et al. 2022

2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)

View full text Add to dashboard Cite

Interconnection networks are the communication backbone of modern high-performance computing systems and an optimised interconnection network is crucial for the performance and utilisation of the system as a whole. One element of the interconnection network is the routing algorithm, which directly influences how we are able to utilise the physical network topology. InfiniBand is one of the most common network architectures used in high-performance computing and traditionally it only supported static routing. For multi-path networks such as Fat-trees, static routing is inefficient because it cannot balance traffic in real-time nor utilise multiple paths efficiently under adversarial traffic. This again potentially leads to unnecessary contention and an underutilised network, which has led to numerous proposals on how to avoid this by using adaptive routing. Adaptive routing has recently been introduced in InfiniBand and in this paper we evaluate to what extent the expected benefits of adaptive routing is true for InfiniBand. Through a set of experiments on HDR InfiniBand equipment we describe the basic behaviour of adaptive routing in InfiniBand, its benefits in Fat tree topologies and the unfortunate side effects related to unfairness that adaptive routing in general might introduce, including such phenomena as the reverse parking lot problem and congestion spreading.

show abstract

Section: Introductionmentioning

confidence: 99%