Host Side Dynamic Reconfiguration with InfiniBand

Guay, Wei Lin; Reinemo, Sven-Arne; Lysne, Olav; Skeie, Tor; Johnsen, Bjorn Dag; Holen, Line

doi:10.1109/cluster.2010.21

Cited by 9 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reason why minhop will not deadlock on a ring constructed out of four nodes is the fact that its implementation in OpenSM assigns symmetric paths for source-destination and destination-source pairs, thus, it breaks the credit loop necessary for a deadlock to occur. However, it is a well known fact that minhop deadlocks on at least a 5-node ring [Guay et al 2010], and any 3-stage fat-tree that has more than one root contains such a ring. Therefore, almost any 3-stage or larger fat-tree routed with minhop will potentially deadlock, thus, it makes our experiment valid and practical.…”

Section: Resultsmentioning

confidence: 99%

“…This example also shows that any 3-stage fat-tree can deadlock with node-to-node and switch-toswitch traffic present, if minhop is run on it. This is because minhop is unable to route a ring of 5 nodes or bigger in a deadlock-free manner [Guay et al 2010], and almost any (apart from a single-root tree) 3-stage fat-tree will contain a 6-node ring. By having shown this deadlock example, we have demonstrated the need for an algorithm that will provide deadlock-free routing in fat-tree topologies when switch-to-switch communication between all switches is present.…”

Section: Switch-to-switch Routingmentioning

confidence: 99%

See 1 more Smart Citation

sFtree

Bogdański

Reinemo

Sem-Jacobsen

et al. 2012

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

Existing fat-tree routing algorithms fully exploit the path diversity of a fat-tree topology in the context of compute node traffic, but they lack support for deadlock-free and fully connected switch-to-switch communication. Such support is crucial for efficient system management, for example, in InfiniBand (IB) systems. With the general increase in system management capabilities found in modern InfiniBand switches, the lack of deadlock-free switch-to-switch communication is a problem for fat-tree-based IB installations because management traffic might cause routing deadlocks that bring the whole system down. This lack of deadlock-free communication affects all system management and diagnostic tools using LID routing.In this paper, we propose the sFtree routing algorithm that guarantees deadlock-free and fully connected switch-to-switch communication in fat-trees while maintaining the properties of the current fat-tree algorithm. We prove that the algorithm is deadlock free and we implement it in OpenSM for evaluation. We evaluate the performance of the sFtree algorithm experimentally on a small cluster and we do a large-scale evaluation through simulations. The results confirm that the sFtree routing algorithm is deadlock-free and show that the impact of switch-to-switch management traffic on the end-node traffic is negligible.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Switch-to-switch Routingmentioning

confidence: 99%

sFtree

Bogdański

Reinemo

Sem-Jacobsen

et al. 2012

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fig. 1 summarises the optimisation feedback cycle that consists of the SM, the PM and our modified host stack with the host side dynamic reconfiguration capability [12,11]. In a subnet, the SM periodically sweeps the subnet to discover changes and maintain a fully connected subnet.…”

Section: Overviewmentioning

confidence: 99%

“…We introduce a performance manager [14] that monitors the network using hardware port counters to detect congestion and optimises the current VL allocation by classifying flows as either slow lane (contributors to congestion) or fast lane (victims of congestion). Then the optimisation is applied using the host side dynamic reconfigration method we proposed in [12]. The effect being that all flows contributing to congestion are migrated to a separate VL (slow lane) in order to avoid the negative impact of head-of-line blocking on the flows not contributing to congestion (victim flows).…”

Section: Introductionmentioning

confidence: 99%

“…As a result, this solution might not be suitable for congestion problem of a more persistent nature because the oscillations that can reduce the overall network throughput. In our previous work [12], we are able to obtain a better overall network throughput in certain congestion scenarios by avoiding the oscillations. Such persistent congestion problems occur when traffic has been moved away from a failed link, when multiple jobs run on the same system, and compete for network resources, or when a system is not balanced for the application that runs on it.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

dFtree

Guay

Reinemo

Lysne

et al. 2011

Proceedings of the First International Workshop on Network-Aware Data Management

Self Cite

View full text Add to dashboard Cite

End-point hotspots can cause major slowdowns in interconnection networks due to head-of-line blocking and congestion. Therefore, avoiding congestion is important to ensure high performance for the network traffic. It is especially important in situations where permanent congestion, which results in permanent slowdown, can occur. Permanent congestion occurs when traffic has been moved away from a failed link, when multiple jobs run on the same system, and compete for network resources, or when a system is not balanced for the application that runs on it.In this paper we suggest a mechanism for dynamic allocation of virtual lanes and live optimization of the distribution of flows between the allocated virtual lanes. The purpose is to alleviate the negative effect of permanent congestion by separating network flows into slow lane and fast lane traffic. Flows destined for a end-point hot-spot is placed in the slow lane and all other flows are placed in the fast lane. Consequently, the flows in the fast lane are unaffected by the head-of-line blocking created by the hot-spot traffic.We demonstrate the feasibility of this approach using a modified version of OFED and OpenSM with fat-tree routing on a small InfiniBand cluster. Our experiments show an increase in throughput ranging from 150% to 468% compared to the conventional fat-tree algorithm in OFED.

show abstract