Efficient network isolation and load balancing in multi-tenant HPC clusters

Zahid, Feroz; Gran, Ernst Gunnar; Bogdański, Bartosz; Johnsen, Bjorn Dag; Skeie, Tor

doi:10.1016/j.future.2016.04.003

Cited by 15 publications

(8 citation statements)

References 42 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Limited members are restricted to communicate only with the full members of the partition. As per the current implementation, the pFTree routing algorithm, as well as its extended versions from Paper III [149], does not support overlapping partitions and limited membership types. Although a node is allowed to be a member of multiple partitions, only one of those partitions can be marked as primary, to be used in pFTree routing.…”

Section: Future Workmentioning

confidence: 98%

“…Moreover, the weighted fat-tree routing and partition-aware routing algorithm, being subjected to different routing criteria, can lead to contradictory routing assignments. The Paper III [149] attack both these issues. We provide an extended version of the pFTree routing algorithm that takes provider-defined tenant-wise isolation policies into consideration when routing the fat-tree topology.…”

Section: Improvements To the Fat-tree Routingmentioning

confidence: 99%

“…Building on the partition-aware routing introduced in Paper II [148], in Paper III [149] we provided two significant extensions to the pFTree routing algorithm for fat-tree topologies.…”

Section: Paper Iii: Efficient Network Isolation and Load Balancing Inmentioning

confidence: 99%

“…III Efficient network isolation and load balancing in multi-tenant HPC clusters[149] Published in Future Generation Computer Systems (FGCS),Volume 72, July 2017. Authors Feroz Zahid, Ernst Gunnar Gran, Bartosz Bogdański, Bjørn Dag Johnsen and Tor Skeie System and method for supporting efficient load-balancing in a high performance computing (hpc) environment.…”

mentioning

confidence: 99%

See 3 more Smart Citations

A Weighted Fat-Tree Routing Algorithm for Efficient Load-Balancing in Infini Band Enterprise Clusters

Zahid

Gran

Bogdański

et al. 2015

2015 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

View full text Add to dashboard Cite

Section: Future Workmentioning

confidence: 98%

Section: Improvements To the Fat-tree Routingmentioning

confidence: 99%

“…Building on the partition-aware routing introduced in Paper II [148], in Paper III [149] we provided two significant extensions to the pFTree routing algorithm for fat-tree topologies.…”

Section: Paper Iii: Efficient Network Isolation and Load Balancing Inmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

A Weighted Fat-Tree Routing Algorithm for Efficient Load-Balancing in Infini Band Enterprise Clusters

Zahid

Gran

Bogdański

et al. 2015

2015 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

View full text Add to dashboard Cite

“…As a result, full utilization of the interconnect is not achieved. Typical challenges in a cloud environment, such as elastic load-balancing, efficient virtualization, and tenant performance isolation, can only be addressed in an IB system when routing is done by taking cloud specific information, for example location of the tenant nodes, into consideration [7].…”

Section: Introductionmentioning

confidence: 99%

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

Zahid

Taherkordi

Gran

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

Clouds offer flexible and economically attractive compute and storage solutions for enterprises. However, the effectiveness of cloud computing for high-performance computing (HPC) systems still remains questionable. When clouds are deployed on lossless interconnection networks, like InfiniBand (IB), challenges related to load-balancing, low-overhead virtualization, and performance isolation hinder full potential utilization of the underlying interconnect. Moreover, cloud data centers incorporate a highly dynamic environment rendering static network reconfigurations, typically used in IB systems, infeasible. In this paper, we present a framework for a self-adaptive network architecture for HPC clouds based on lossless interconnection networks, demonstrated by means of our implemented IB prototype. Our solution, based on a feedback control and optimization loop, enables the lossless HPC network to dynamically adapt to the varying traffic patterns, current resource availability, workload distributions, and also in accordance with the service provider-defined policies. Furthermore, we present IBAdapt, a simplified ruled-based language for the service providers to specify adaptation strategies used by the framework. Our developed self-adaptive IB network prototype is demonstrated using state-of-the-art industry software. The results obtained on a test cluster demonstrate the feasibility and effectiveness of the framework when it comes to improving Quality-of-Service compliance in HPC clouds.

show abstract

Network slicing to improve multicasting in HPC clusters

Alsmadi

Khreishah

2018

Cluster Comput

View full text Add to dashboard Cite

Efficient network isolation and load balancing in multi-tenant HPC clusters

Cited by 15 publications

References 42 publications

A Weighted Fat-Tree Routing Algorithm for Efficient Load-Balancing in Infini Band Enterprise Clusters

A Weighted Fat-Tree Routing Algorithm for Efficient Load-Balancing in Infini Band Enterprise Clusters

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

Network slicing to improve multicasting in HPC clusters

Contact Info

Product

Resources

About