Adaptive Speculation for Efficient Internetware Application Execution in Clouds

Abstract

Modern Cloud computing systems are massive in scale, featuring environments that can execute highly dynamic Internetware applications with huge numbers of interacting tasks. This has led to a substantial challenge − the straggler problem, whereby a small subset of slow tasks significantly impede parallel job completion. This problem results in longer service responses, degraded system performance, and late timing failures that can easily threaten Quality of Service (QoS) compliance. Speculative execution (or speculation) is the prominent method deployed in Clouds to tolerate stragglers by creating task replicas at runtime. The method detects stragglers by specifying a predefined threshold to calculate the difference between individual tasks and the average task progression within a job. However, such a static threshold debilitates speculation effectiveness as it fails to capture the intrinsic diversity of timing constraints in Internetware applications, as well as dynamic environmental factors such as resource utilization. By considering such characteristics, different levels of strictness for replica creation can be imposed to adaptively achieve specified levels of QoS for different applications. In this paper we present an algorithm to improve the execution efficiency of Internetware applications by dynamically calculating the straggler threshold, considering key parameters including job QoS timing constraints, task execution progress, and optimal system resource utilization. We implement this dynamic straggler threshold into the YARN architecture to evaluate it's effectiveness against existing state-of-the-art solutions. Results demonstrate that the proposed approach is capable of reducing parallel job response times by up to 20% compared to the static threshold, as well as a higher speculation success rate, achieving up to 66.67% against 16.67% in comparison to the static method.

Facilitating Dynamic RT-QoS for Massive-Scale Autonomous Cyber-Physical Systems

“…Representative speculative based variations include LATE [28], Mantri [29], Dolly [30], Adaptive Speculator [7], SkewTune [31], CREST [32], etc. Each of the related work has its own characteristic in terms of suitable target environments or straggler types.…”

“…Current parallel jobs normally uses Progress Score (PS) of a task and corresponding elapsed time (t −t 0 ) when calculating the estimated completion using Equation 1 [28] [7], where PS is given in systems such as Hadoop and YARN (PS t i j is the PS for the i th task in parallel job j at time stamp t). But they ignore the influence brought by resource availability and node heterogeneity, which should provide additional knowledge for a more accurate prediction of task completion time.…”

“…Then if the T T F is greater than the required deadline D an alert is provided with the resource configuration that would be required to meet the deadline, as shown in Algorithm 3. The alert is then passed onto the speculation manager which creates appropriate replicas [7].…”

“…Therefore in order to develop techniques to handle the explosion of the big data streams [5] in a timely fashion new techniques bringing together real-time stream processing [6], straggler mitigation [7], and QoS prediction [8] must developed. Furthermore with the emergence of the IoS paradigm where simulations are deployed as intelligence services (SIMaaS) [2] interacting with in-the-loop systems -such as hardware, model, or human -the Service Level Agreement (SLA) and QoS must provide real-time guarantees.…”

Prism

“…Stragglers are highly transient and stochastic in nature, making it difficult to reproduce system conditions leading to their occurrence. Whilst we have begun to empirically study and understand straggler root causes [3] [8], it is unknown to what degree system conditions directly influence their manifestation.…”