TetriSched

Tumanov, Alexey; Zhu, Timothy; Park, Jun Woo; Kozuch, Michael A.; Harchol-Balter, Mor; Ganger, Gregory R.

doi:10.1145/2901318.2901355

Cited by 121 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For resource management systems more broadly, Paragon [25] and Quasar [26] use collaborative filtering to match workloads to different machine types and avoid interference; their goal is complementary to Decima's. Tetrisched [74], like Decima, plans ahead in time, but uses a constraint solver to optimize job placement and requires the user to supply explicit constraints with their jobs. Firmament [33] also uses a constraint solver and achieves high-quality placements, but requires an administrator to configure an intricate scheduling policy.…”

Section: Related Workmentioning

confidence: 99%

Learning scheduling algorithms for data processing clusters

Mao¹,

Schwarzkopf²,

Venkatakrishnan³

et al. 2019

Proceedings of the ACM Special Interest Group on Data Communication

475

292

View full text Add to dashboard Cite

Efficiently scheduling data processing jobs on distributed compute clusters requires complex algorithms. Current systems use simple, generalized heuristics and ignore workload characteristics, since developing and tuning a scheduling policy for each workload is infeasible. In this paper, we show that modern machine learning techniques can generate highly-efficient policies automatically.Decima uses reinforcement learning (RL) and neural networks to learn workload-specific scheduling algorithms without any human instruction beyond a high-level objective, such as minimizing average job completion time. However, off-the-shelf RL techniques cannot handle the complexity and scale of the scheduling problem. To build Decima, we had to develop new representations for jobs' dependency graphs, design scalable RL models, and invent RL training methods for dealing with continuous stochastic job arrivals.Our prototype integration with Spark on a 25-node cluster shows that Decima improves average job completion time by at least 21% over hand-tuned scheduling heuristics, achieving up to 2× improvement during periods of high cluster load.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning scheduling algorithms for data processing clusters

Mao¹,

Schwarzkopf²,

Venkatakrishnan³

et al. 2019

Proceedings of the ACM Special Interest Group on Data Communication

475

292

View full text Add to dashboard Cite

show abstract

“…Tetrisched [30] is a heterogeneity-aware scheduler, that leverages runtime estimates, generated from information collected by the reservation system, to plan ahead. Task requirements are captured using a declarative language known as Space-Time Request Language.…”

Section: Constraint-aware Schedulingmentioning

confidence: 99%

Constraint-Aware Federated Scheduling for Data Center Workloads

Thiyyakat,

Kalambur,

Sitaram

2023

IoT

View full text Add to dashboard Cite

The use of data centers is ubiquitous, as they support multiple technologies across domains for storing, processing, and disseminating data. IoT applications utilize both cloud data centers and edge data centers based on the nature of the workload. Due to the stringent latency requirements of IoT applications, the workloads are run on hardware accelerators such as FPGAs and GPUs for faster execution. The introduction of such hardware alongside existing variations in the hardware and software configurations of the machines in the data center, increases the heterogeneity of the infrastructure. Optimal job performance necessitates the satisfaction of task placement constraints. This is accomplished through constraint-aware scheduling, where tasks are scheduled on worker nodes with appropriate machine configurations. The presence of placement constraints limits the number of suitable resources available to run a task, leading to queuing delays. As federated schedulers have gained prominence for their speed and scalability, we assess the performance of two such schedulers, Megha and Pigeon, within a constraint-aware context. We extend our previous work on Megha by comparing its performance with a constraint-aware version of the state-of-the-art federated scheduler Pigeon, PigeonC. The results of our experiments with synthetic and real-world cluster traces show that Megha reduces the 99th percentile of job response time delays by a factor of 10 when compared to PigeonC. We also describe enhancements made to Megha’s architecture to improve its scheduling efficiency.

show abstract

“…Heterogeneity-aware resource managers Gavel [58] (✓) 𝑃 (✓) 𝐸,𝑆 AlloX [44] (✓) 𝑃 (✓) 𝐸,𝑆 Gandiva [81] (✓) 𝑃 (✓) 𝐸,𝑆 Themis [53] (✓) 𝑃 (✓) 𝐸,𝑆 (✓) 𝐴 Tetrished [75] (✓) 𝑃 (✓) 𝑆 (✓) 𝐴…”

Section: Challenges and System Designmentioning

confidence: 99%

“…Management of resources considering end hosts/servers in data centers (DCs) without taking into account INC is already a nontrivial problem which has been heavily investigated over the past years [6,9,14,23,34,59,75,77], also considering GPUs and other accelerators [29,54,58,60,81]. Throwing network appliance resources Ð INC resources for short Ð into the mix adds new challenges and significantly exacerbates existing ones (see Fig.…”

Section: Introductionmentioning

confidence: 99%

Switches for HIRE: resource scheduling for data center in-network computing

Blöcher

Wang

Eugster

et al. 2021

Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

TetriSched

Cited by 121 publications

References 36 publications

Learning scheduling algorithms for data processing clusters

Learning scheduling algorithms for data processing clusters

Constraint-Aware Federated Scheduling for Data Center Workloads

Switches for HIRE: resource scheduling for data center in-network computing

Contact Info

Product

Resources

About