Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud

Jackson, Keith; Ramakrishnan, Lavanya; Muriki, Krishna; Canon, Shane; Cholia, Shreyas; Shalf, John; Wasserman, Harvey J.; Wright, Nicholas J.

doi:10.1109/cloudcom.2010.69

Cited by 479 publications

(318 citation statements)

References 26 publications

(28 reference statements)

Supporting

Mentioning

299

Contrasting

Unclassified

Order By: Relevance

“…StarCluster creates Amazon EC2 instance for master and all the nodes. It enables SSH access between them [11] the memory blocks between them are shared by NFS.…”

Section: Methodsmentioning

confidence: 99%

Data Mining Using Hierarchical Agglomerative Clustering Algorithm in Distributed Cloud Computing Environment

Srivastava¹,

Shah²,

Valia³

et al. 2013

IJCTE

View full text Add to dashboard Cite

Abstract-Today increase in worldwide business led to offices distributed across geographical location .Hence data are loosely distributed across regionalized large scale databases across regionalized offices. To perform data mining it is required to merge distributed data and perform data mining algorithm on it. Cloud computing poses a diversity of challenges in data mining operation arising out of the dynamic structure of data distribution as against the use of typical database scenarios in conventional architecture. This document presents a way to implement Hierarchical Agglomerative Clustering Algorithm in such way so as to make it suitable for large dataset and increase its efficiency by executing task in parallel. The result shows that with increase in data set linear growth of execution time.Index Terms-Star cluster, hierarchal agglomerative clustering, virtual k mean, cloud computing.

show abstract

“…StarCluster creates Amazon EC2 instance for master and all the nodes. It enables SSH access between them [11] the memory blocks between them are shared by NFS.…”

Section: Methodsmentioning

confidence: 99%

Data Mining Using Hierarchical Agglomerative Clustering Algorithm in Distributed Cloud Computing Environment

Srivastava¹,

Shah²,

Valia³

et al. 2013

IJCTE

View full text Add to dashboard Cite

show abstract

“…They conclude that the cost and performance of the services vary significantly depending on the workload. Jackson et al [Jackson, Ramakrishnan, Muriki, Canon, Cholia, Shalf, Wasserman and Wright, 2010;Jackson, Ramakrishnan, Runge and Thomas, 2010] port various scientific applications, such as SNFactory pipeline, to Amazon [EC2, 2014]. Their results show that the performance of EC2 is more variable and slower than non-cloud computing platforms, due to the limitation of interconnects on EC2.…”

Section: ¾º½ ê ôð ø óò ó ë öú × ò ðóù ×mentioning

confidence: 99%

“…Performance variability: Various studies [Kossmann et al, 2010;Jackson, Ramakrishnan, Runge and Thomas, 2010;Jackson, Ramakrishnan, Muriki, Canon, Cholia, Shalf, Wasserman and Wright, 2010;Ueda and Nakatani, 2010;Nurmi et al, 2009] present performance studies and report their experiences on migration of various applications to commercial cloud platforms. A common observation is the high variability in the quality of service.…”

Section: ¾º½ ê ôð ø óò ó ë öú × ò ðóù ×mentioning

confidence: 99%

Resource management of replicated service systems provisioned in the cloud

Björkqvist

Birke

Binder

2016

NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium

View full text Add to dashboard Cite

I certify that except where due acknowledgement has been given, the work presented in this thesis is that of the author alone; the work has not been submitted previously, in whole or in part, to qualify for any other academic award; and the content of the thesis is the result of work which has been carried out since the official commencement date of the approved research program. Lugano, 11 February 2015 ×ØÖ Ø Service providers seek scalable and cost-effective cloud solutions for hosting their applications. Despite significant recent advances facilitating the deployment and management of services on cloud platforms, a number of challenges still remain. Service providers are confronted with time-varying requests for the provided applications, inter-dependencies between different components, performance variability of the procured virtual resources, and cost structures that differ from conventional data centers. Moreover, fulfilling service level agreements, such as the throughput and response time percentiles, becomes of paramount importance for ensuring business advantages. Mathias BjörkqvistIn this thesis, we explore service provisioning in clouds from multiple points of view. The aim is to best provide service replicas in the form of VMs to various service applications, such that their tail throughput and tail response times, as well as resource utilization, meet the service level agreements in the most cost effective manner. In particular, we develop models, algorithms and replication strategies that consider multi-tier composed services provisioned in clouds. We also investigate how a service provider can opportunistically take advantage of observed performance variability in the cloud. Finally, we provide means of guaranteeing tail throughput and response times in the face of performance variability of VMs, using Markov chain modeling and large deviation theory. We employ methods from analytical modeling, event-driven simulations and experiments. Overall, this thesis provides not only a multi-faceted approach to exploring several crucial aspects of hosting services in clouds, i.e., cost, tail throughput, and tail response times, but our proposed resource management strategies are also rigorously validated via trace-driven simulation and extensive experiments.

show abstract

“…In addition, the aim has been to evaluate the performance gap between cloud and on-premise resources [9][10][11][12]. Even though cloud typically has slower internal network speeds than onpremise resources, bursting jobs to the cloud can still provide better overall performance in overloaded environments.…”

Section: Introductionmentioning

confidence: 99%

Job placement advisor based on turnaround predictions for HPC hybrid clouds

Cunha¹,

Rodrigues²,

Tizzei³

et al. 2017

Future Generation Computer Systems

View full text Add to dashboard Cite

Abstract-Several companies and research institutes are moving their CPU-intensive applications to hybrid High Performance Computing (HPC) cloud environments. Such a shift depends on the creation of software systems that help users decide where a job should be placed considering execution time and queue wait time to access on-premise clusters. Relying blindly on turnaround prediction techniques will affect negatively response times inside HPC cloud environments. This paper introduces a tool to make job placement decisions in HPC hybrid cloud environments taking into account the inaccuracy of execution and waiting time predictions. We used job traces from real supercomputing centers to run our experiments, and compared the performance between environments using real speedup curves. We also extended a stateof-the-art machine learning based predictor to work with data from the cluster scheduler. Our main findings are: (i) depending on workload characteristics, there is a turning point where predictions should be disregarded in favor of a more conservative decision to minimize job turnaround times and (ii) scheduler data plays a key role in improving predictions generated with machine learning using job trace data-our experiments showed around 20% prediction accuracy improvements.

show abstract

Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud

Cited by 479 publications

References 26 publications

Data Mining Using Hierarchical Agglomerative Clustering Algorithm in Distributed Cloud Computing Environment

Data Mining Using Hierarchical Agglomerative Clustering Algorithm in Distributed Cloud Computing Environment

Resource management of replicated service systems provisioned in the cloud

Job placement advisor based on turnaround predictions for HPC hybrid clouds

Contact Info

Product

Resources

About