Mystic: Predictive Scheduling for GPU Based Cloud Servers Using Machine Learning

Ukidave, Yash; Li, Xiangyu; Kaeli, David

doi:10.1109/ipdps.2016.73

Cited by 47 publications

(17 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some works use collaborative filtering to colocate tasks in clouds by estimating application interference [30]. Others are closer to the application level and use binary classification to distinguish benign memory faults from application errors in order to execute recovery algorithms (see [31] for instance).…”

Section: Data-aware Resource Managementmentioning

confidence: 99%

Tuning EASY-Backfilling Queues

Lelong

Reis

Trystram

2018

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. EASY-Backfilling is a popular scheduling heuristic for allocating jobs in large scale High Performance Computing platforms. While its aggressive reservation mechanism is fast and prevents job starvation, it does not try to optimize any scheduling objective per se. We consider in this work the problem of tuning EASY using queue reordering policies. More precisely, we propose to tune the reordering using a simulationbased methodology. For a given system, we choose the policy in order to minimize the average waiting time. This methodology departs from the First-Come, First-Serve rule and introduces a risk on the maximum values of the waiting time, which we control using a queue thresholding mechanism. This new approach is evaluated through a comprehensive experimental campaign on five production logs. In particular, we show that the behavior of the systems under study is stable enough to learn a heuristic that generalizes in a train/test fashion. Indeed, the average waiting time can be reduced consistently (between 11% to 42% for the logs used) compared to EASY, with almost no increase in maximum waiting times. This work departs from previous learning-based approaches and shows that scheduling heuristics for HPC can be learned directly in a policy space.

show abstract

Section: Data-aware Resource Managementmentioning

confidence: 99%

Tuning EASY-Backfilling Queues

Lelong

Reis

Trystram

2018

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Chen et al [7] classify and predict the duration of different GPU tasks and provide QoS support to the concurrent GPU applications by resource reservation. Ukidave et al [40] exploit machine learning to identify the similarities between the arriving kernels and the running kernels and use this technique to avoid QoS violations in a GPU-equipped cluster. Zhang et al [8] propose a runtime system that exploits the newly added spatial multitasking feature in a GPU and raises the accelerator utilization while achieving the latency targets for user-facing services.…”

Section: Related Workmentioning

confidence: 99%

Toward OS-Level and Device-Level Cooperative Scheduling for Multitasking GPUs

et al. 2020

View full text Add to dashboard Cite

As one of the most popular accelerators, the graphics processing unit (GPU) has been extensively adopted throughout the world. With the burst of new applications and the growing scale of data, co-running applications on limited GPU resources has become increasingly important due to its dramatic improvement in overall system efficiency. Quality of service (QoS) support among concurrent generalpurpose GPU (GPGPU) applications is currently one of the most trending research topics. Prior efforts have been focused on providing QoS support either with OS-level or device-level scheduling methods. Each of these scheduling methods possesses pros and cons and may be unable to independently cover all the scheduling cases. In this paper, we propose a cooperative QoS scheduling scheme (C-QoS) that consists of operating-system-level (OS-level) scheduling and device-level scheduling. Our proposed scheme can control the progress of a kernel and provide thorough QoS support for concurrent applications in multitasking GPUs. Due to the accurate resource management of the copy engine and execution engine, C-QoS achieves QoS goals 23.33% more often than state-of-the-art QoS support mechanisms. The results demonstrate that cooperative methods achieve 17.27% higher system utilization than uncooperative methods. INDEX TERMS Multitasking, parallel architectures, quality of service.

show abstract

“…Chen et al [19] propose a task duration predictor and a task reordering mechanism based on the predictions to guarantee QoS. Ukidave et al [12] present an interferenceaware mechanism for co-scheduling on GPUs based on machine learning to predict whether kernels can share a GPU efficiently. Wen et al [20] propose a graph-based algorithm to schedule kernels in pairs.…”

Section: Related Workmentioning

confidence: 99%

“…We propose a novel graph-based preemptive coscheduling solution, that defines the kernels submission order focusing on reducing the number of preemptions. The idea is to exploit the kernels interference profile provided by previous works analysis of how the kernels resource usage impacts the interference in their co-execution [11], [12]. More formally, we deal with the following problem: given a co-scheduling speedup matrix S, where S i,j is the speed at which kernel i makes progress when running with j, and the duration of each kernel, what is the best way to co-schedule them, in order to minimize their makespan.…”

Section: Introductionmentioning

confidence: 99%

Algorithms for Preemptive Co-scheduling of Kernels on GPUs

Eyraud-Dubois

Bentes

2020

2020 IEEE 27th International Conference on High Performance Computing, Data, and Analytics (HiPC)

View full text Add to dashboard Cite

Modern GPUs allow concurrent kernel execution and preemption to improve hardware utilization and responsiveness. Currently, the decision on the simultaneous execution of kernels is performed by the hardware, which can lead to unreasonable use of resources. In this work, we tackle the problem of co-scheduling for GPUs in high competition scenarios. We propose a novel graphbased preemptive co-scheduling algorithm, with the focus on reducing the number of preemptions. We show that the optimal preemptive makespan can be computed by solving a Linear Program in polynomial time. Based on this solution we propose graph theoretical model and an algorithm to build preemptive schedules which minimizes the number of preemptions. We show, however, that finding the minimal amount of preemptions among all preemptive solutions of optimal makespan is a NP-hard problem. We performed experiments on real-world GPU applications and our approach can achieve optimal makespan by preempting 6 to 9% of the tasks.

show abstract

Mystic: Predictive Scheduling for GPU Based Cloud Servers Using Machine Learning

Cited by 47 publications

References 23 publications

Tuning EASY-Backfilling Queues

Tuning EASY-Backfilling Queues

Toward OS-Level and Device-Level Cooperative Scheduling for Multitasking GPUs

Algorithms for Preemptive Co-scheduling of Kernels on GPUs

Contact Info

Product

Resources

About