Falcon

Abstract

We present Falcon, a novel scheduler design for large scale data analytics workloads. To improve the quality of the scheduling decisions, Falcon uses a single central scheduler. To scale the central scheduler to support large clusters, Falcon offloads the scheduling operation to a programmable switch. The core of the Falcon design is a novel pipeline-based scheduling logic that can schedule tasks at line-rate. Our prototype evaluation on a cluster with a Barefoot Tofino switch shows that the proposed approach can reduce scheduling overhead by 26 times and increase the scheduling throughput by 25 times compared to state-of-the-art centralized and decentralized schedulers. CCS Concepts • Networks → Network services → In-network processing; • Networks → Network services → Cloud computing;

Network-accelerated cluster scheduler

“…P4-K8s-Scheduler achieves median task placement latency of ∼50 𝜇s with 1,000-machine switch support, and ∼170 ms total delay for 1,000 scheduling requests. The scheduling overhead has been reduced by an order of magnitude compared to state-of-the-art Kubernetes schedulers [13], and by up to 50% compared to other networkaccelerated schedulers [16].…”