Straggler Detection in Parallel Computing Systems through Dynamic Threshold Calculation

Abstract

-Cloud computing systems face the substantial challenge of the Long Tail problem: a small subset of straggling tasks significantly impede parallel jobs completion. This behavior results in longer service response times and degraded system utilization. Speculative execution, which create task replicas at runtime, is a typical method deployed in large-scale distributed systems to tolerate stragglers. This approach defines stragglers by specifying a static threshold value, which calculates the temporal difference between an individual task and the average task progression for a job. However, specifying static threshold debilitates speculation effectiveness as it fails to consider the intrinsic diversity of job timing constraints within modern day Cloud computing systems. Capturing such heterogeneity enables the ability to impose different levels of strictness for replica creation while achieving specified levels of QoS for different application types. Furthermore, a static threshold also fails to consider system environmental constraints in terms of replication overheads and optimal system resource usage. In this paper we present an algorithm for dynamically calculating a threshold value to identify task stragglers, considering key parameters including job QoS timing constraints, task execution characteristics, and optimal system resource utilization. We study and demonstrate the effectiveness of our algorithm through simulating a number of different operational scenarios based on real production cluster data against state-of-the-art solutions. Results demonstrate that our approach is capable of creating 58.62% less replicas under high resource utilization while reducing response time up to 17.86% for idle periods compared to a static threshold.

Straggler Root-Cause and Impact Analysis for Massive-scale Virtualized Cloud Datacenters

“…50% greater than median task execution, DoS-Index ≥ 2.5), as tasks that complete just below the threshold will be detected as false positives, however still impede job execution completion. These results indicate the need for more intelligent metrics for straggler detection -transitioning away from a fixed temporal boundary as defined in [4][8] towards an adaptive boundary that consider metrics such as task progression, system conditions and job QoS as detailed in [34]. We observe that the online analytics agent produce approximately 0.2% CPU usage for both job types, representing a fractional amounts of server usage, and observing no indication of causing an increment in straggler behavior caused by high CPU.…”

An investigation into the impacts of task-level behavioural heterogeneity upon energy efficiency in Cloud datacentres

“…But tasks in a parallel job are usually independent to each other, though long tails do not directly affect the progress rate of the other colocated tasks, long running tasks still delay the job completion to an irresistible margin. Most of the existing literature adopts a threshold value of 50% [3] to classify long tails, whereby tasks exhibiting a runtime duration 50% more than the average job duration are classified as stragglers, as shown in equation (8). In general, the identification efficiency of long tails depends on the time of classification.…”

“…Users submit their request in the form of jobs [1] at the datacentres, a single job may encompass one to several number of tasks. The scheduler [2] in the datacentre schedules and allocates the tasks belonging to a single job across the available server nodes based on the computational requirements of every individual tasks, unlike a typical Map Reduce platform [3] divides the job into multiple tasks for executing the tasks in an evenly distributed subsets. In other words, tasks belonging to a single job show increased level of heterogeneity [4,5] in terms of their resource requirements, resource consumption and task duration etc., but a Map Reduce platform tries to achieve even execution profiles across the distributed tasks such as similar duration, resource consumption etc.…”

“…Since resource level breaches are one of the major causes for task terminations, tasks characterising higher resource requirements are vulnerable for terminations potentially causing task-level stragglers. Existing works of straggler identification addresses task duration as the only major attribute to label tasks as stragglers, reprocessing a few tasks within jobs resulting from tasks terminations have been ignored in straggler analysis [3,11,12] which leave the impacts of considerable proportions of tasks-level stragglers unnoticed. Whilst stragglers are approached to mitigate long tails for shorten durations, this paper addresses the phenomenon of task behavioural heterogeneity in terms of their resource intensiveness as one of the major causes for extravagant provisioning of resources.…”

An Approach for Modeling and Ranking Node-Level Stragglers in Cloud Datacenters

“…In most literature pertaining to stragglers, this threshold is typically configured to a value of 1.5 (i.e. tasks whose execution is 50% greater than the average execution of tasks within the same job) [10][14] [18], while [19] proposes a dynamic threshold calculation algorithm to define task stragglers in accordance to workload type and cluster resource usage.…”