Constructing virtual 5‐dimensional tori out of lower‐dimensional network cards

Abstract

In the Top500 and Graph500 lists of the last years, some of the most powerful systems implement a torus topology to interconnect the millions of computing nodes they include. Some of these torus networks are of five or six dimensions, which implies an additional difficulty as the node degree increases. In previous works, we proposed and evaluated the nD Twin (nDT) torus topology to virtually increase the dimensions a torus is able to implement. We showed that this new topology reduces the distances between nodes, increasing, therefore, global network performance. In this work, we present how to build a 5DT torus network using a specific commercial 6-port network card (EXTOLL card) to interconnect those nodes. We show, using the same number of cards, that the performance of the 5DT torus network we are able to implement using our proposal is higher than the performance of the 3D torus network for the same number of compute nodes.

Trends in High‐Performance Interconnection Networks in the Exascale and Big‐Data Era (HiPINEB 2017)

“…Benito et al explore a range of routing solutions based on exploiting explicit congestion notification messages (in particular, 802.1Qau) to adapt the number of packets using non‐minimal paths. Andujar et al present how to build a 5DT torus network using a specific commercial 6‐port network card (EXTOLL) to interconnect the system nodes. Tasoulas et al suggest solutions to rectify the space‐domain scalability issues that are present in vSwitch‐enabled subnets as a result of the VMs using dedicated layer‐two addresses, also discussing routing strategies for virtualized environments using vSwitches and presenting a routing algorithm for Fat‐Trees.…”