Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

Bryk, Piotr; Malawski, Maciej; Juve, Gideon; Deelman, Ewa

doi:10.1007/s10723-015-9355-6

Cited by 38 publications

(18 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If such VM is not available, a new VM instance is created. In this algorithm, file transfers take zero time; (ii) Storage-Aware Static Provisioning Static Scheduling (SA-SPSS) [44], it is a modified version of the original SPSS algorithm to operate in environments where file transfers take non-zero time. It handles file transfers between tasks.…”

Section: Evaluation Methodsmentioning

confidence: 99%

Data-Aware Scheduling Strategy for Scientific Workflow Applications in IaaS Cloud Computing

Makhlouf¹,

Yagoubi²

2019

IJIMAI

View full text Add to dashboard Cite

Scientific workflows benefit from the cloud computing paradigm, which offers access to virtual resources provisioned on pay-as-you-go and on-demand basis. Minimizing resources costs to meet user's budget is very important in a cloud environment. Several optimization approaches have been proposed to improve the performance and the cost of data-intensive scientific Workflow Scheduling (DiSWS) in cloud computing. However, in the literature, the majority of the DiSWS approaches focused on the use of heuristic and metaheuristic as an optimization method. Furthermore, the tasks hierarchy in data-intensive scientific workflows has not been extensively explored in the current literature. Specifically, in this paper, a data-intensive scientific workflow is represented as a hierarchy, which specifies hierarchical relations between workflow tasks, and an approach for data-intensive workflow scheduling applications is proposed. In this approach, first, the datasets and workflow tasks are modeled as a conditional probability matrix (CPM). Second, several data transformation and hierarchical clustering are applied to the CPM structure to determine the minimum number of virtual machines needed for the workflow execution. In this approach, the hierarchical clustering is done with respect to the budget imposed by the user. After data transformation and hierarchical clustering, the amount of data transmitted between clusters can be reduced, which can improve cost and makespan of the workflow by optimizing the use of virtual resources and network bandwidth. The performance and cost are analyzed using an extension of Cloudsim simulation tool and compared with existing multi-objective approaches. The results demonstrate that our approach reduces resources cost with respect to the user budgets.

show abstract

Section: Evaluation Methodsmentioning

confidence: 99%

Data-Aware Scheduling Strategy for Scientific Workflow Applications in IaaS Cloud Computing

Makhlouf¹,

Yagoubi²

2019

IJIMAI

View full text Add to dashboard Cite

show abstract

“…Types of cost [113] DATA RENTING COST. [114] DATA STORAGE, DATA TRANSFER COST. [86,115,116] COMPUTATION, TRANSFER COST.…”

Section: Approachmentioning

confidence: 99%

Task scheduling in cloud computing based on metaheuristic techniques: A review paper

Al-Arasi¹,

Saif²

2020

EAI Endorsed Transactions on Cloud Systems

View full text Add to dashboard Cite

Cloud computing delivers computing resources like software and hardware as a service to the users through a network. Due to the scale of the modern datacentres and their dynamic resources provisioning nature, we need efficient scheduling techniques to manage these resources. The main objective of scheduling is to assign tasks to adequate resources in order to achieve one or more optimization criteria. Scheduling is a challenging issue in the cloud environment, therefore many researchers have attempted to explore an optimal solution for task scheduling in the cloud environment. They have shown that traditional scheduling is not efficient in solving this problem and produce an optimal solution with polynomial time in the cloud environment. However, they introduced sub-optimal solutions within a short period of time. Meta-heuristic techniques have provided near-optimal or optimal solutions within an acceptable time for such problems. In this work, we have introduced the major concepts of resource scheduling and provided a comparative analysis of many task scheduling techniques based on different optimization criteria.

show abstract

“…Unfortunately, to achieve these features a high penalty is present, especially when taking into account the storage performance, which can become something not tolerable for Big Data applications that require to process huge amounts of data e ciently or within time limits. Several studies exist [7,36] that describe how the penalty imposed by remote or virtualized storage access (typically the case when the cloud is used) severely a↵ects data-intensive applications. In [36], the authors provide an in-depth study of the challenges of data-intensive computing in the cloud.…”

Section: Cloud-based Architectures and Servicesmentioning

confidence: 99%

Big Data-Oriented PaaS Architecture with Disk-as-a-Resource Capability and Container-Based Virtualization

et al. 2018

View full text Add to dashboard Cite

With the increasing adoption of Big Data technologies as basic tools for the ongoing Digital Transformation, there is a high demand for data-intensive applications. In order to e ciently execute such applications, it is vital that cloud providers change the way hardware infrastructure resources are managed to improve their performance. However, the increasing use of virtualization technologies to achieve an e cient usage of infrastructure resources continuously widens the gap between applications and the underlying hardware, thus decreasing resource e ciency for the end user. Moreover, this scenario is especially troublesome for Big Data applications, as storage resources are one of the most heavily virtualized, thus imposing a significant overhead for large-scale data processing. This paper proposes a novel PaaS architecture specifically oriented for Big Data where the scheduler o↵ers disks as resources alongside the more common CPU and memory resources, looking forward to provide a better storage solution for the user. Furthermore, virtualization overheads are reduced to the bare minimum by replacing heavy hypervisor-based technologies with operating-system-level virtualization based on light software containers. This architecture has been deployed on a Big Data infrastructure at the CESGA supercomputing center, used as a testbed to compare its performance with OpenStack, a popular private cloud platform. Results have shown significant performance improvements, reducing the execution time of representative Big Data workloads by up to 4.5x.

show abstract

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

Cited by 38 publications

References 46 publications

Data-Aware Scheduling Strategy for Scientific Workflow Applications in IaaS Cloud Computing

Data-Aware Scheduling Strategy for Scientific Workflow Applications in IaaS Cloud Computing

Task scheduling in cloud computing based on metaheuristic techniques: A review paper

Big Data-Oriented PaaS Architecture with Disk-as-a-Resource Capability and Container-Based Virtualization

Contact Info

Product

Resources

About