Chronos: Failure-aware scheduling in shared Hadoop clusters

Abstract

Hadoop emerged as the de facto state-of-the-art system for MapReduce-based data analytics. The reliability of Hadoop systems depends in part on how well they handle failures. Currently, Hadoop handles machine failures by re-executing all the tasks of the failed machines (i.e., executing recovery tasks). Unfortunately, this elegant solution is entirely entrusted to the core of Hadoop and hidden from Hadoop schedulers. The unawareness of failures therefore may prevent Hadoop schedulers from operating correctly towards meeting their objectives (e.g., fairness, job priority) and can significantly impact the performance of MapReduce applications. This paper presents Chronos, a failure-aware scheduling strategy that enables an early yet smart action for fast failure recovery while still operating within a specific scheduler objective. Upon failure detection, rather than waiting an uncertain amount of time to get resources for recovery tasks, Chronos leverages a lightweight preemption technique to carefully allocate these resources. In addition, Chronos considers data locality when scheduling recovery tasks to further improve the performance. We demonstrate the utility of Chronos by combining it with Fifo and Fair schedulers. The experimental results show that Chronos recovers to a correct scheduling behavior within a couple of seconds only and reduces the job completion times by up to 55% compared to state-of-the-art schedulers.

Enabling fast failure recovery in shared Hadoop clusters: Towards failure-aware scheduling

“…Relationship to previous work. This paper extends our previous contribution introduced in a previous paper [8] by providing more detailed descriptions and more thorough experiments. In particular, we have substantially extended two sections: While Section 2 gives an overview of MapReduce, Hadoop, scheduling in Hadoop and its fault-tolerance mechanism, Section 8 discusses related works on scheduling, failure recovery, task preemption and data-aware task scheduling in MapReduce.…”

Hadoop MapReduce scheduling paradigms

“…Almost all the surveyed schedulers in this paper have advantages in terms of fairness and completion time compared to the default Hadoop scheduling policy. Interestingly, FRESH [10] and COSHH-hybrid [8] have the potential to become a native part of Hadoop, replacing FIFO and Fair sharing, as well as Chronos [11] which holds a lot of promise while still needing further testing. When it comes to large enterprise environments, LsPS [15] represents a promising approach as it delivered unprecedented performance and user control in a scalable and dynamic cluster, vastly improving upon default schedulers.…”

“…2) Chronos: Instead of creating a totally new "default" scheduler from scratch, a different approach for enhancing the native Hadoop scheduler is proposed by the authors of Chronos: Failure-Aware Scheduling in Shared Hadoop Clusters [11]. The authors argue that the performance of Hadoop systems in part depends on how failures are handled.…”

Handbook of Research on Cloud Computing and Big Data Applications in IoT

“…In case of hardware or software failures, the affected hardware that might be prone to failure can be avoided in scheduling to avoid any further failures. Chronos (Yildiz et al 2015) is a Hadoop-based failure-aware scheduler that uses pre-emption on failed jobs. Then it recovers from failure by reallocating the failed jobs with pre-empted resources to meet the SLA objectives.…”