TAGO: Rethinking Routing Design in High Performance Reconfigurable Networks

Teh, Min Yee; Hung, Yu‐Han; Yan, Shijia; Glick, Madeleine; Shalf, John; Bergman, Keren

doi:10.1109/sc41405.2020.00029

Cited by 6 publications

(4 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Routing. The routing used on a PRN is based on the trafficaware, globally-direct oblivious (TAGO) routing scheme in [64], which sends intra-pod traffic using equal cost multi-path (ECMP) and inter-pod traffic using direct and indirect podto-pod routes. Packets that traverse indirect pod-to-pod routes will first be routed to a randomly chosen intermediate pod, before being forwarded to their destination pod.…”

Section: A Methodologymentioning

confidence: 99%

“…These experiments are implemented in Python 2.7, and run on a desktop with 2.6 GHz dual Intel i7 cores with 16 GB of RAM. For the PRN, we use Gurobi [63] to solve a relaxed linear programming (LP) that maximizes throughput [64]. For the TRN, we implement two variants of matching algorithms: (1) a centralized max-weight matching and (2) a distributed Gale-Shapley matching based on [15].…”

Section: Demand-aware Processing Overheadmentioning

confidence: 99%

See 1 more Smart Citation

Performance trade-offs in reconfigurable networks for HPC

Teh

Glick

et al. 2022

J. Opt. Commun. Netw.

View full text Add to dashboard Cite

Designing efficient interconnects to support high-bandwidth and low-latency communication is critical toward realizing high performance computing (HPC) and data center (DC) systems in the exascale era. At extreme computing scales, providing the requisite bandwidth through overprovisioning becomes impractical. These challenges have motivated studies exploring reconfigurable network architectures that can adapt to traffic patterns at runtime using optical circuit switching. Despite the plethora of proposed architectures, surprisingly little is known about the relative performances and trade-offs among different reconfigurable network designs. We aim to bridge this gap by tackling two key issues in reconfigurable network design. First, we study how cost, power consumption, network performance, and scalability vary based on optical circuit switch (OCS) placement in the physical topology. Specifically, we consider two classes of reconfigurable architectures: one that places OCSs between top-of-rack (ToR) switches—ToR-reconfigurable networks (TRNs)—and one that places OCSs between pods of racks—pod-reconfigurable networks (PRNs). Second, we tackle the effects of reconfiguration frequency on network performance. Our results, based on network simulations driven by real HPC and DC workloads, show that while TRNs are optimized for low fan-out communication patterns, they are less suited for carrying high fan-out workloads. PRNs exhibit better overall trade-off, capable of performing comparably to a fully non-blocking fat tree for low fan-out workloads, and significantly outperform TRNs for high fan-out communication patterns.

show abstract

Section: A Methodologymentioning

confidence: 99%

Section: Demand-aware Processing Overheadmentioning

confidence: 99%

Performance trade-offs in reconfigurable networks for HPC

Teh

Glick

et al. 2022

J. Opt. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…COUDER relies on robust traffic engineering to deliver good performance. In contrast, many prior works on optical circuit-switched data centers performs traffic engineering based only on a single predicted traffic matrix [3], [4], [10], [60]. However, accurate traffic prediction is difficult, and an inaccurate prediction may lead to poor performance.…”

Section: B Traffic Engineeringmentioning

confidence: 99%

Enabling Quasi-Static Reconfigurable Networks With Robust Topology Engineering

Teh

Zhao

Cao

et al. 2023

IEEE/ACM Trans. Networking

Self Cite

View full text Add to dashboard Cite

Many optical circuit switched data center networks (DCN) have been proposed in the last decade to attain higher capacity and topology reconfigurability, though commercial adoption of these architectures have been minimal. One major challenge these architectures face is the difficulty of handling uncertain traffic demands using commercial optical circuit switches (OCS) with high switching latency. Prior works have generally focused on developing fast-switching OCS prototypes to quickly react to traffic variations through frequent reconfigurations. This approach, however, adds tremendous complexity overhead to the control plane, and raises the barrier for commercial adoption of optical circuit switched data center networks. We propose COUDER, a robust topology and routing optimization framework for reconfigurable optical circuit switched data centers. COUDER co-optimizes topology and routing based on a convex set of traffic matrices, and offers strict throughput guarantees for any future traffic matrices bounded by the convex set. For the bursty traffic demands that are unbounded by the convex set, we employ a desensitization technique to reduce performance hit. This enables COUDER to generate topology and routing solutions capable of handling unexpected traffic changes without relying on frequent topology reconfigurations. Our extensive evaluations based on Facebook's production DCN traces show that, even with daily reconfigurations which could be realized by current commercial MEMS-based OCSs from Calient Technologies, COUDER achieves about 20% lower max link utilization, and about 32% lower average hop count compared to cost-equivalent static topologies. Our work shows that adoption of reconfigurable topologies in commercial DCNs is feasible even without fast OCSs.

show abstract

“…Another design is traffic-aware topology + traffic-agnostic routing. TAGO [29] reconfigures PoD-level topology based on traffic patterns, and performs ECMP for intra-PoD routing and topology-aware VLB for inter-block routing, i.e., for any inter-PoD packet, the probability that it chooses an intermediate PoD is proportional to the number of links between its source PoD and the intermediate PoD. However, as we will show in Fig.…”

Section: B Traffic Aware Vs Agnostic Designsmentioning

confidence: 99%

Threshold-Based Routing-Topology Co-Design for Optical Data Center

Cao

Zhao

Dai

et al. 2023

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Despite the bandwidth scaling limit of electrical switching and the high cost of building Clos data center networks (DCNs), the adoption of optical DCNs is still limited. There are two reasons. First, existing optical DCN designs usually face high deployment complexity. Second, these designs are not full-optical and the performance benefit over the non-blocking Clos DCN is not clear. After exploring the design tradeoffs of the existing optical DCN designs, we propose TROD (Threshold Routing based Optical Datacenter), a low-complexity optical DCN with superior performance than other optical DCNs. There are two novel designs in TROD that contribute to its success. First, TROD performs robust topology optimization based on the recurring traffic patterns and thus does not need to react to every traffic change, which lowers deployment and management complexity. Second, TROD introduces tVLB (threshold-based Valiant Load Balance), which can avoid network congestion as much as possible even under unexpected traffic bursts. We conduct simulation based on both Facebook's real DCN traces and our synthesized highly bursty DCN traces. TROD reduces flow completion time (FCT) by about 1.15-2.16× compared to Google's Jupiter DCN, at least 2× compared to other optical DCN designs, and about 2.4-3.2× compared to expander graph DCN. Compared with the non-blocking Clos, TROD reduces the hop count of the majority packets by one, and could even outperform the non-blocking Clos with proper bandwidth over-provision at the optical layer. Note that TROD can be built with commercially available hardware and does not require host modifications.

show abstract

TAGO: Rethinking Routing Design in High Performance Reconfigurable Networks

Cited by 6 publications

References 55 publications

Performance trade-offs in reconfigurable networks for HPC

Performance trade-offs in reconfigurable networks for HPC

Enabling Quasi-Static Reconfigurable Networks With Robust Topology Engineering

Threshold-Based Routing-Topology Co-Design for Optical Data Center

Contact Info

Product

Resources

About