Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds

Malawski, Maciej; Juve, Gideon; Deelman, Ewa; Nabrzyski, Jarek

doi:10.1109/sc.2012.38

Cited by 199 publications

(173 citation statements)

References 19 publications

Supporting

Mentioning

173

Contrasting

Order By: Relevance

“…Methodology The simulator in [5], originally developed in [17], was used to implement and evaluate the two proposed cost-aware algorithms. Resources that operate at a range of frequencies between a minimum and maximum frequency, f min =1000MHz and f max =3000MHz, with a frequency step ( f step ) of 100MHz are assumed.…”

Section: Experimental Evaluation and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Cost-Efficient CPU Provisioning for Scientific Workflows on Clouds

Pietri

Sakellariou

2016

Economics of Grids, Clouds, Systems, and Services

View full text Add to dashboard Cite

Abstract. Cloud providers now offer resources as combinations of CPU frequencies and prices, with faster resources (which operate at higher frequencies) charged at a higher monetary cost. With the emergence of this new pricing scheme, the problem of choosing cost-efficient configurations is becoming even more challenging for users. The frequencies required to achieve cost-efficient configurations may vary in different scenarios, depending on both the provider's pricing model and the application characteristics. In this paper, two cost-aware algorithms that select low-cost CPU frequencies for each resource to complete a scientific workflow application within a deadline and at a minimum cost are presented. The proposed approaches are evaluated and compared through simulation using different pricing models that charge resource provisioning also based on the CPU frequency.

show abstract

Section: Experimental Evaluation and Resultsmentioning

confidence: 99%

“…Existing work on task scheduling and resource provisioning addresses various optimization goals in different computing systems such as grids [1,9,20,22] and clouds [3,11,17]. Execution time and cost optimization is the focus in many studies [14,25,27].…”

Section: Related Workmentioning

confidence: 99%

Cost-Efficient CPU Provisioning for Scientific Workflows on Clouds

Pietri

Sakellariou

2016

Economics of Grids, Clouds, Systems, and Services

View full text Add to dashboard Cite

show abstract

“…For scientific workflows, the following works were proposed; for example, on a CPU node [37], in the Grid environment [11] [16], and in the cloud environment [26] [25]. However, the large scientific workflows are frequently running on a large computer with sophisticated storage and networking resources that are not easily captured by the existing models.…”

Section: Related Workmentioning

confidence: 99%

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Yoo

Koo

Cao

et al. 2016

Conquering Big Data With High Performance Computing

View full text Add to dashboard Cite

Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe-art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.

show abstract

“…Malawski et al [101] consider how to optimally provision for multiple a priori known workflow ensembles with varying deadline and budget constraints. However, they do not consider the network as a critical part of the topology orchestration and focus only on the workflow model rather than…”

Section: Related Work On Topology Management Of Cloud-distributed Appmentioning

confidence: 99%

Service level agreements-driven management of distributed applications in cloud computing environments

Antonescu

Braun

2015

2015 IFIP/IEEE International Symposium on Integrated Network Management (IM)

View full text Add to dashboard Cite

Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds

Cited by 199 publications

References 19 publications

Cost-Efficient CPU Provisioning for Scientific Workflows on Clouds

Cost-Efficient CPU Provisioning for Scientific Workflows on Clouds

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Service level agreements-driven management of distributed applications in cloud computing environments

Contact Info

Product

Resources

About