Resilient co-scheduling of malleable applications

Abstract

Recently, the benefits of co-scheduling several applications have been demonstrated in a fault-free context, both in terms of performance and energy savings. However, large-scale computer systems are confronted by frequent failures, and resilience techniques must be employed for large applications to execute efficiently. Indeed, failures may create severe imbalance between applications, and significantly degrade performance. In this paper, we aim at minimizing the expected completion time of a set of co-scheduled applications. We propose to redistribute the resources assigned to each application upon the occurrence of failures, and upon the completion of some applications, in order to achieve this goal. First, we introduce a formal model and establish complexity results. The problem is NP-complete for malleable applications, even in a faultfree context. Therefore, we design polynomial-time heuristics that perform redistributions and account for processor failures. A fault simulator is used to perform extensive simulations that demonstrate the usefulness of redistribution and the performance of the proposed heuristics.

List and shelf schedules for independent parallel tasks to minimize the energy consumption with discrete or continuous speeds

“…• ∀i, (w i,j ) j is non-decreasing in j (when using more processors, there is more overhead due to the parallelization, which is a common assumption [9,8]).…”

Shelf schedules for independent moldable tasks to minimize the energy consumption

“…• ∀i, (w i,j ) j is non-decreasing in j (when using more processors, there is more overhead due to the parallelization, which is a common assumption [8,7]).…”

Combining malleability and I/O control mechanisms to enhance the execution of multiple applications

“…Iserte et al (2017) integrate the Nanos++ runtime with Slurm RMS for increasing the system throughput via a better resource utilization. Benoit et al (2018) introduce a model for scheduling malleable applications on a failureprone platform. The novelty of this work is to consider the effect of failures in the applications on the scheduling policy.…”