Approximation Proofs of a Fast and Efficient List Scheduling Algorithm for Task-Based Runtime Systems on Multicores and GPUs

Abstract

In High Performance Computing, heterogeneity is now the norm with specialized accelerators like GPUs providing efficient computational power. The added complexity has led to the development of task-based runtime systems, which allow complex computations to be expressed as task graphs, and rely on scheduling algorithms to perform load balancing between all resources of the platforms. Developing good scheduling algorithms, even on a single node, and analyzing them can thus have a very high impact on the performance of current HPC systems. The special case of two types of resources (namely CPUs and GPUs) is of practical interest. HeteroPrio is such an algorithm which has been proposed in the context of fast multipole computations, and then extended to general task graphs with very interesting results. In this paper, we provide a theoretical insight on the performance of HeteroPrio, by proving approximation bounds compared to the optimal schedule in the case where all tasks are independent and for different platform sizes. Interestingly, this shows that spoliation allows to prove approximation ratios for a list scheduling algorithm on two unrelated resources, which is not possible otherwise. We also establish that almost all our bounds are tight. Additionally, we provide an experimental evaluation of HeteroPrio on real task graphs from dense linear algebra computation, which highlights the reasons explaining its good practical performance.

Online Scheduling of Task Graphs on Hybrid Platforms

“…Low complexity algorithms, which are closer to our work, have been studied in [5,11] and achieve approximation ratios respectively equal to 2 and 2 + √ 2. For tasks with precedence constraints, Kedad-Sidhoum et al [18] provided a tight 6-approximation algorithm based on linear programming.…”

“…Allowing task spoliation (where tasks can be canceled and restarted on any resource, as done in [5]) does not help, and allowing task migration (where tasks can be preempted and resumed on any resource) only halves the bounds. Table 1 summarizes the lower bounds obtained for all combination of knowledge given to the scheduler and flexibility on the task processing (for proofs, please refer to [10]).…”

“…Schedulers for independent tasks on hybrid platforms have first been proposed [5,8,11]. Some of them have been adapted for task graphs: [20] extends the algorithm of [5] to the (offline) scheduling of task graphs, while [2] adapts an online scheduler for independent tasks on hybrid platforms [17] to obtain a competitive online scheduler for task graphs.…”

Low-Cost Approximation Algorithms for Scheduling Independent Tasks on Hybrid Platforms

“…The closest approaches in terms of cost, behavior and guarantee are HeteroPrio [8], a (2+ √ 2)-approximation, and CLB2C [17], a 2-approximation assuming that no single is greater than the optimal makespan. They both start by sorting the tasks by their acceleration ratios.…”

“…In this context, low complexity schedulers have recently been proposed. The closest approaches to our work in terms of cost, behavior, and guarantee are HeteroPrio [8], a (2 + √ 2)-approximation algorithm, and CLB2C [17], a 2-approximation algorithm in the case where every task processing time, on any resource, is smaller than the optimal makespan.…”