Minimizing average coflow completion time with decentralized scheduling

Luo, Shouxi; Yu, Hongfang; Zhao, Yangming; Wu, Bin; Wang, Sheng; Li, Le Min

doi:10.1109/icc.2015.7248339

Cited by 26 publications

(5 citation statements)

References 10 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep-Aalo [20] updates the thresholds of queues in Alao by deep reinforcement learning. Dogar [21], Luo [22], and D-CAS [23] study coflow-aware scheduling scheme in a distributed manner. Further, CODA [24] proposes an algorithm of automatically assigning flows to coflows before scheduling.…”

Section: A Related Workmentioning

confidence: 99%

Joint Coflow Optimization for Data Center Networks

2021

IEEE Access

View full text Add to dashboard Cite

Data parallel applications in data centers generate, process, and store huge volumes of data. Coflow Completion time (CCT) is one of the major performance metrics to capture application-level semantics. This paper is the first one to study the joint consideration of task placement, coflow bandwidth scheduling, and path choice to minimize the average CCT in intra-data center. This paper proposes a joint online scheduling framework, which first develops a 2-approximation algorithm to reduce the CCT of a single coflow, and then follows the Shortest Remaining Time First (SRTF) principle to schedule multiple coflows. Extensive simulations based on practical trace demonstrate that the proposed framework has better performance than the state-of-the-art works.INDEX TERMS Data center, coflow scheduling, task placement, path choice.

show abstract

Section: A Related Workmentioning

confidence: 99%

Joint Coflow Optimization for Data Center Networks

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…CODA [23] employs an algorithm to automatically divide network flows into different coflows before scheduling, such that extract coflows can be done without modification. Dogar [24] and Luo [25] investigate the decentralized coflow-aware scheduling problem.…”

Section: Related Workmentioning

confidence: 99%

Joint Online Coflow Optimization Across Geo-Distributed Datacenters

2020

IEEE Access

View full text Add to dashboard Cite

Growing data volumes have been generated, stored and processed across geographically distributed datacenters. Processing such geo-distributed datasets should consider task or job placement, heterogeneous available link bandwidth, and routing decisions. Thus optimizing the transmissions of inter-datacenter flows, especially coflows that capture application-level semantics, is important. However, prior solutions on coflow scheduling either merely ignore routing or fix the endpoint placement. This article focuses on the problem of jointly considering endpoint placement, coflow scheduling, and coflow routing to minimize the average coflow completion time (CCT) across geo-distributed datacenters. This paper first presents the system model and problem formulation, then derives a randomized approximation algorithm for a single coflow. After that it also proposes an online algorithm to handle the complex cases of multiple coflows. Through theoretical analysis, a proof that the proposed algorithms have a non-trivial competitive ratio is given. Results from extensive simulations demonstrate that the proposed algorithms can significantly reduce the CCT of coflows and at the same time, have similar algorithm run time, compared with the state-of-the-art solutions. INDEX TERMS Scheduling algorithms, routing, task placement, performance analysis.

show abstract

“…In [5], the Varys scheme improves the performance of [8] by leveraging more sophisticated heuristics such as "smallestbottleneck-first" and "smallest-total-size-first", where global information about coflows is required. The D-CAS scheme in [11] exploits a similar "shortest-remaining-time-first" principle for coflow scheduling. The Aalo framework [6] generalizes the classic least-attained service (LAS) discipline [9] to coflow scheduling; such a scheme does not require prior knowledge about coflows.…”

Section: Related Workmentioning

confidence: 99%

Coflow scheduling in input-queued switches: Optimal delay scaling and algorithms

Liang

Modiano

2017

IEEE INFOCOM 2017 - IEEE Conference on Computer Communications

View full text Add to dashboard Cite

Abstract-A coflow is a collection of parallel flows belonging to the same job. It has the all-or-nothing property: a coflow is not complete until the completion of all its constituent flows. In this paper, we focus on optimizing coflow-level delay, i.e., the time to complete all the flows in a coflow, in the context of an N × N input-queued switch. In particular, we develop a throughput-optimal scheduling policy that achieves the best scaling of coflow-level delay as N → ∞. We first derive lower bounds on the coflow-level delay that can be achieved by any scheduling policy. It is observed that these lower bounds critically depend on the variability of flow sizes. Then we analyze the coflow-level performance of some existing coflow-agnostic scheduling policies and show that none of them achieves provably optimal performance with respect to coflow-level delay. Finally, we propose the Coflow-Aware Batching (CAB) policy which achieves the optimal scaling of coflow-level delay under some mild assumptions.

show abstract

Minimizing average coflow completion time with decentralized scheduling

Cited by 26 publications

References 10 publications

Joint Coflow Optimization for Data Center Networks

Joint Coflow Optimization for Data Center Networks

Joint Online Coflow Optimization Across Geo-Distributed Datacenters

Coflow scheduling in input-queued switches: Optimal delay scaling and algorithms

Contact Info

Product

Resources

About