Integrating microsecond circuit switching into the data center

Porter, George; Strong, Richard; Farrington, Nathan; Forencich, Alex; Chen-Sun, Pang; Rosing, Tajana; Fainman, Yeshaiahu; Papen, George C.; Vahdat, Amin

doi:10.1145/2486001.2486007

Cited by 181 publications

(55 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Known traffic patterns are also a hallmark of many data center applications, in which hybrid use of optical circuits can complement packet switching [8]. Our approach avoids being limited to such environments.…”

Section: Discussionmentioning

confidence: 99%

“…This is only possible if data exchange patterns are both known in advance and are stable over long periods. Such patterns currently occur in datacenters and other researchers are developing optical circuit systems for that environment [8]. We prefer to focus on the more general, most flexible solution of optical packet switching, which would support not only these known, long-scale traffic patterns but also the unknown, rapidly shifting patterns prominent elsewhere throughout the Internet.…”

Section: Our Assumptionsmentioning

confidence: 99%

“…This is rarely the case for Internet traffic in general, but can be used for provisioning links between Internet routers in the Internet core (where traffic is large enough and where its backbone patterns are predictable in the aggregate), especially over long timescales. Known traffic patterns are also a hallmark of many data center applications, in which hybrid use of optical circuits can complement packet switching [8]. Our approach avoids being limited to such environments.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Design for an Internet Router with a Digital Optical Data Plane

et al. 2017

View full text Add to dashboard Cite

This paper presents a complete design for an optical Internet router based on the component steps required for Internet protocol (IP) packet forwarding. Implementations of hop count decrement and header matching are integrated with a simulation-based approach to variable-length packet traffic merging that avoids recirculation, demonstrating an approach for an all-optical data plane. A method for IPv4 checksum computation is introduced, and this and previously designed components are extended from binary to higher-density (multiple bits per symbol) encodings. The implications of this design are considered, including the potential for chip-level and system integration, as well as the requirements of basic optical processing components.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Our Assumptionsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Design for an Internet Router with a Digital Optical Data Plane

et al. 2017

View full text Add to dashboard Cite

show abstract

“…On the micro level, flow-based optimization mechanisms (e.g., [4]- [8]) attempt to minimize average completion time of flows or groups of flows by exploiting flow size provided by the applications. On the macro level, architectural bandwidth provisioning (e.g., [9]- [12]) and traffic engineering (e.g., [13]- [15]) solutions try to estimate * This work was performed when Hao Wang and Ziyang Li were intern students at SING Group @ HKUST.…”

Section: Introductionmentioning

confidence: 99%

FLOWPROPHET: Generic and Accurate Traffic Prediction for Data-Parallel Cluster Computing

Wang

Chen

et al. 2015

2015 IEEE 35th International Conference on Distributed Computing Systems

View full text Add to dashboard Cite

Abstract-Data-parallel computing frameworks (DCF) such as MapReduce, Spark, and Dryad etc. have tremendous applications in big data and cloud computing, and throw tons of flows into data center networks. In this paper, we design and implement FLOWPROPHET, a general framework to predict traffic flows for DCFs.To this end, we analyze and summarize the common features of popular DCFs, and gain a key insight: since application logic in DCFs is naturally expressed by directed acyclic graphs (DAG), DAG contains necessary time and data dependencies for accurate flow prediction. Based on the insight, FLOWPROPHET extracts DAGs from user applications, and uses the time and data dependencies to calculate flow information 4-tuple, (source, destination, flow_size, establish_time), ahead-oftime for all flows. We also provide generic programming interface to FLOWPROPHET, so that current and future DCFs can deploy FLOWPROPHET readily. We implement FLOWPROPHET on both Spark and Hadoop, and perform extensive evaluations on a testbed with 37 physical servers. Our implementation and experiments demonstrate that, with time in advance and minimal cost, FLOWPROPHET can achieve almost 100% accuracy in source, destination, and flow size predictions. With accurate prediction from FLOWPROPHET, the job completion time of a Hadoop TeraSort benchmark is reduced by 12.52% on our cluster with a simple network scheduler.

show abstract

“…Fast optical switching technologies have become available that improve upon the 3D-MEMS OCS switch [8][9][10][11][12]. These switches can be built using technologies such as arrayed waveguide grating routers (AWGRs), semiconductor optical amplifiers (SOAs), 1 × N photonic switches and wavelength selective switches (WSSs).…”

Section: Introductionmentioning

confidence: 99%

Software-Controlled Next Generation Optical Circuit Switching for HPC and Cloud Computing Datacenters

et al. 2015

View full text Add to dashboard Cite

In this paper, we consider the performance of optical circuit switching (OCS) systems designed for data center networks by using network-level simulation. Recent proposals have used OCS in data center networks but the relatively slow switching times of OCS-MEMS switches (10-100 ms) and the latencies of control planes in these approaches have limited their use to the largest data center networks with workloads that last several seconds. Herein, we extend the applicability and generality of these studies by considering dynamically changing short-lived circuits in software-controlled OCS switches, using the faster switching technologies that are now available. The modelled switch architecture features fast optical switches in a single hop topology with a centralized, software-defined optical control plane. We model different workloads with various traffic aggregation parameters to investigate the performance of such designs across usage patterns. Our results show that, with suitable choices for the OCS system parameters, delay performance comparable to that of electrical data center networks can be obtained.

show abstract

Integrating microsecond circuit switching into the data center

Cited by 181 publications

References 17 publications

A Design for an Internet Router with a Digital Optical Data Plane

A Design for an Internet Router with a Digital Optical Data Plane

FLOWPROPHET: Generic and Accurate Traffic Prediction for Data-Parallel Cluster Computing

Software-Controlled Next Generation Optical Circuit Switching for HPC and Cloud Computing Datacenters

Contact Info

Product

Resources

About