A scalable, commodity data center network architecture

Abstract

Today's data centers may contain tens of thousands of computers with significant aggregate bandwidth requirements. The network architecture typically consists of a tree of routing and switching elements with progressively more specialized and expensive equipment moving up the network hierarchy. Unfortunately, even when deploying the highest-end IP switches/routers, resulting topologies may only support 50% of the aggregate bandwidth available at the edge of the network, while still incurring tremendous cost. Nonuniform bandwidth among data center nodes complicates application design and limits overall system performance.In this paper, we show how to leverage largely commodity Ethernet switches to support the full aggregate bandwidth of clusters consisting of tens of thousands of elements. Similar to how clusters of commodity computers have largely replaced more specialized SMPs and MPPs, we argue that appropriately architected and interconnected commodity switches may deliver more performance at less cost than available from today's higher-end solutions. Our approach requires no modifications to the end host network interface, operating system, or applications; critically, it is fully backward compatible with Ethernet, IP, and TCP.

The influence of datacenter usage on symmetry in datacenter network design

“…Fat-Tree [3], VL2 [28], and Portland [52]. It is generally acknowledged that this basic switch-centric architecture, where routing is handled by routing tables within the switches (and so there can be no direct server-to-server links), has its limitations as datacenters grow (see, for example, [51]).…”

Dahu: Commodity switches for direct connect data center networks

“…Routing is typically integrated with proprietary protocols in NICs and switches. Data center networks on the other hand are built using low cost commodity offthe-shelf switches [2,16]. So commodity Ethernet switch silicon must be enhanced to provide the necessary features to support direct connect topologies in the data center while keeping costs low.…”

“…There have been many recent proposals for scale-out multipath data center topologies such as Clos networks [2,16,23], direct networks like HyperX [1], Flattened Butterfly [20], DragonFly [21], and even randomly connected topologies proposed in Jellyfish [27]. Many current proposals use ECMP-based techniques which are inadequate to utilize all paths, or to dynamically load balance traffic.…”

“…For hierarchical topologies like Fattrees [2], shortest path routing has been largely sufficient when there are no failures. However, recently proposed direct network topologies like HyperX, BCube, and Flattened Butterfly [1,17,20], which employ paths of different lengths, have not seen adoption in data centers due to the limitations imposed by commodity data center switches and shortest path routing.…”

CLON: Overlay Networks and Gossip Protocols for Cloud Environments

“…In a smaller scope, this can be also observed in the architecture of a single data center, as collections of nodes are also grouped in a federated manner. The increasing aggregate bandwidth demand could be alleviated, but not solved, by using a fat tree network layout [2], where leaf nodes are grouped in a way to mitigate the load imposed on the individual network devices, while at the same time providing transparent load balancing and failover capabilities among those devices.…”