Amazon S3 for science grids

Palankar, Mayur; Iamnitchi, Adriana; Ripeanu, Matei; Garfinkel, Simson L.

doi:10.1145/1383519.1383526

Cited by 354 publications

(178 citation statements)

References 4 publications

Supporting

Mentioning

173

Contrasting

Order By: Relevance

“…In the course of their experiments, they also had to restart about one out of ten runs due to the occurrence of failures. Similar observations have been made in other studies on the performance of cloud infrastructure [10,11].…”

Section: The Need For Introducing Instabilitysupporting

confidence: 89%

“…We therefore set the default I/O throughput of virtual machines to 20 MB/s. The external bandwidth of virtual machines was set to 0.25 MB/s, based on the remote access performance of S3 reported by Palankar et al [11] and Pelletingeas [37].…”

Section: Experimental Settingsmentioning

confidence: 99%

“…The effect of these factors of uncertainty and instability is not negligible and has been repeatedly observed to strongly influence the runtime of a given application on commercial clouds such as Amazon's Elastic Compute Cloud (EC2) (e.g., [7][8][9][10][11][12][13]). …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DynamicCloudSim: Simulating heterogeneity in computational clouds

Bux

Leser

2015

Future Generation Computer Systems

View full text Add to dashboard Cite

Simulation has become a commonly employed first step in evaluating novel approaches towards resource allocation and task scheduling on distributed architectures. However, existing simulators fall short in their modeling of the instability common to shared computational infrastructure, such as public clouds. In this work, we present DynamicCloudSim which extends the popular simulation toolkit CloudSim with several factors of instability, including inhomogeneity and dynamic changes of performance at runtime as well as failures during task execution. As a use case and validation of the introduced functionality, we simulate the impact of instability on scientific workflow scheduling by assessing and comparing the performance of four schedulers in the course of several experiments. Results indicate that our model seems to adequately capture the most important aspects of cloud performance instability, though a validation on real hardware is still pending. The source code of DynamicCloudSim and the examined schedulers is available at https://code.google.com/p/dynamiccloudsim/.

show abstract

Section: The Need For Introducing Instabilitysupporting

confidence: 89%

Section: Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

DynamicCloudSim: Simulating heterogeneity in computational clouds

Bux

Leser

2015

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…Closest to our work, our own [8,16,17] and related [44][45][46]5] studies of multiple scheduling policies have emphasized the inability of any single policy to perform well under a wide yet realistic variety of scientific workloads.…”

Section: Related Workmentioning

confidence: 94%

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

Deng

Verboon

Ren

et al. 2014

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. The popularity of data centers in scientific computing has led to new architectures, new workload structures, and growing customerbases. As a consequence, the selection of efficient scheduling algorithms for the data center is an increasingly costlier and more difficult challenge. To address this challenge, and contrasting previous work on scheduling for scientific workloads, we focus in this work on portfolio schedulinghere, the dynamic selection and use of a scheduling policy, depending on the current system and workload conditions, from a portfolio of multiple policies. We design a periodic portfolio scheduler for the workload of the entire data center, and equip it with a portfolio of resource provisioning and allocation policies. Through simulation based on real and synthetic workload traces, we show evidence that portfolio scheduling can automatically select the scheduling policy to match both user and data center objectives, and that portfolio scheduling can perform well in the data center, relative to its constituent policies.

show abstract

“…Some of previous solutions (such as Chord [8]) with pure decentralized storage models focused on the concept of distributed hash tables (DHT), which assumes the possession of an identifier that identifies the service to be discovered. Some other decentralized approaches, such as Amazon Simple Storage Service (Amazon S3 [9]), focused on a peer-to-peer file distribution protocol called Bittorent, which is designed to distribute large amounts of widely distributed data onto peers, where each peer is capable of requesting and transferring data. Despite of their important features, previous metadata discovery solutions do not address the application requirements of aforementioned target application use domains.…”

Section: Introductionmentioning

confidence: 99%

High‐performance hybrid information service architecture

Aktaş

Pierce

2010

Concurrency and Computation

View full text Add to dashboard Cite

Abstract. We introduce a distributed high performance hybrid Information Service Architecture, which forms a metadata replica hosting system to manage both highlydynamic, small-scale and relatively-large, static metadata associated to Grid/Web Services. We present an empirical evaluation of the proposed architecture and investigate its practical usefulness. The results demonstrate that the proposed system achieves highperformance and fault-tolerance with negligible processing overheads. The results also indicate that efficient decentralized Information Service Architectures can be built by utilizing publish-subscribe based messaging schemes.

show abstract

Amazon S3 for science grids

Cited by 354 publications

References 4 publications

DynamicCloudSim: Simulating heterogeneity in computational clouds

DynamicCloudSim: Simulating heterogeneity in computational clouds

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

High‐performance hybrid information service architecture

Contact Info

Product

Resources

About