Measuring Burstiness in Data Center Applications

Woodruff, Jackson; Moore, Andrew W.; Zilberman, Noa

doi:10.1145/3375235.3375240

Cited by 14 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Distributed ML traffic: There is a wealth of recent studies on profiling DNN training [43]- [49] and reducing communication times [50]- [53], however, a systematic study of the communication overhead and traffic patterns of DNN training does not exist. To the best of our knowledge, there is only one prior measurement study of distributed ML traffic [54], with traces available in [55]. These measurements are performed on a testbed similar to Fig.…”

Section: Related Workmentioning

confidence: 99%

HLS: A Packet Scheduler for Hierarchical Fairness

Luangsomboon

Liebeherr

2021

2021 IEEE 29th International Conference on Network Protocols (ICNP)

View full text Add to dashboard Cite

Traffic from distributed training of machine learning (ML) models makes up a large and growing fraction of the traffic mix in enterprise data centers. While work on distributed ML abounds, the network traffic generated by distributed ML has received little attention. Using measurements on a testbed network, we investigate the traffic characteristics generated by the training of the ResNet-50 neural network with an emphasis on studying its shortterm burstiness. For the latter we propose metrics that quantify traffic burstiness at different time scales. Our analysis reveals that distributed ML traffic exhibits a very high degree of burstiness on short time scales, exceeding a 60:1 peak-to-mean ratio on time intervals as long as 5 ms. We observe that training software orchestrates transmissions in such a way that burst transmissions from different sources within the same application do not result in congestion and packet losses. An extrapolation of the measurement data to multiple applications underscores the challenges of distributed ML traffic for congestion and flow control algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

HLS: A Packet Scheduler for Hierarchical Fairness

Luangsomboon

Liebeherr

2021

2021 IEEE 29th International Conference on Network Protocols (ICNP)

View full text Add to dashboard Cite

show abstract

“…Rack-scale traffic is ON/OFF traffic [19] We believe this trend will continue with network traffic generated by accelerators. Measuring traffic-demands in such environments is hard, let alone learning about workload-characteristics; traffic demands at ns-timescales will be different compared to 𝜇s timescales and ms-timescales [59]. Workload churn and different application mixes adds to the unpredictability.…”

Section: Rack-scale Characteristics and Implicationsmentioning

confidence: 99%

PL2: Towards Predictable Low Latency in Rack-Scale Networks

Le,

Mysore,

Suresh

et al. 2021

Preprint

View full text Add to dashboard Cite

High performance rack-scale offerings package disaggregated pools of compute, memory and storage hardware in a single rack to run diverse workloads with varying requirements, including applications that need low and predictable latency. The intra-rack network is typically high speed Ethernet, which can suffer from congestion leading to packet drops and may not satisfy the stringent tail latency requirements for some workloads (including remote memory/storage accesses). In this paper, we design a Predictable Low Latency (PL2) network architecture for rack-scale systems with Ethernet as interconnecting fabric. PL2 leverages programmable Ethernet switches to carefully schedule packets such that they incur no loss with NIC and switch queues maintained at small, near-zero levels. In our 100 Gbps rack-prototype, PL2 keeps 99th-percentile memcached RPC latencies under 60 µs even when the RPCs compete with extreme offered-loads of 400%, without losing traffic. Network transfers for a machine learning training task complete 30% faster than a receiverdriven scheme implementation modelled after Homa (222ms vs 321ms 99%ile latency per iteration).

show abstract

“…Similarly, stringent performance requirements are introduced by today's trend of resource disaggregation in datacenters where fast access to remote resources (e.g., GPUs or memory) is pivotal for the overall system performance [36]. Building systems with strict performance requirements is especially challenging under bursty traffic patterns as they are commonly observed in datacenter networks [12,16,47,53,55].…”

Section: Introductionmentioning

confidence: 99%

PowerTCP: Pushing the Performance Limits of Datacenter Networks

Addanki¹,

Michel²,

Schmid³

2021

Preprint

View full text Add to dashboard Cite

Increasingly stringent throughput and latency requirements in datacenter networks demand fast and accurate congestion control. We observe that the reaction time and accuracy of existing datacenter congestion control schemes are inherently limited. They either rely only on explicit feedback about the network state (e.g., queue lengths in DCTCP) or only on variations of state (e.g., RTT gradient in TIMELY). To overcome these limitations, we propose a novel congestion control algorithm, POWERTCP, which achieves much more fine-grained congestion control by adapting to the bandwidth-window product (henceforth called power). POWERTCP leverages in-band network telemetry to react to changes in the network instantaneously without loss of throughput and while keeping queues short. Due to its fast reaction time, our algorithm is particularly well-suited for dynamic network environments and bursty traffic patterns. We show analytically and empirically that POWERTCP can significantly outperform the state-ofthe-art in both traditional datacenter topologies and emerging reconfigurable datacenters where frequent bandwidth changes make congestion control challenging. In traditional datacenter networks, POWERTCP reduces tail flow completion times of short flows by 80% compared to DCQCN and TIMELY, and by 33% compared to HPCC even at 60% network load. In reconfigurable datacenters, POWERTCP achieves 85% circuit utilization without incurring additional latency and cuts tail latency by at least 2x compared to existing approaches.

show abstract

Measuring Burstiness in Data Center Applications

Cited by 14 publications

References 10 publications

HLS: A Packet Scheduler for Hierarchical Fairness

HLS: A Packet Scheduler for Hierarchical Fairness

PL2: Towards Predictable Low Latency in Rack-Scale Networks

PowerTCP: Pushing the Performance Limits of Datacenter Networks

Contact Info

Product

Resources

About