Dryad

Isard, Michael; Budiu, Mihai; Yu, Yuan; Birrell, Andrew; Fetterly, Dennis

doi:10.1145/1272996.1273005

Cited by 1,447 publications

(78 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There is already some promising work addressing this problem [14], [15], [20], and we believe the techniques proposed here can be extended to provide an alternative perspective to this challenge. As yet another direction, our techniques can be adapted to work in platforms such as MapReduce/Hadoop [18] and Dryad [19].…”

Section: Related Workmentioning

confidence: 99%

Learning-based Query Performance Modeling and Prediction

Akdere

Çetintemel

Riondato

et al. 2012

2012 IEEE 28th International Conference on Data Engineering

156

196

View full text Add to dashboard Cite

Abstract-Accurate query performance prediction (QPP) is central to effective resource management, query optimization and query scheduling. Analytical cost models, used in current generation of query optimizers, have been successful in comparing the costs of alternative query plans, but they are poor predictors of execution latency. As a more promising approach to QPP, this paper studies the practicality and utility of sophisticated learningbased models, which have recently been applied to a variety of predictive tasks with great success, in both static (i.e., fixed) and dynamic query workloads.We propose and evaluate predictive modeling techniques that learn query execution behavior at different granularities, ranging from coarse-grained plan-level models to fine-grained operatorlevel models. We demonstrate that these two extremes offer a tradeoff between high accuracy for static workload queries and generality to unforeseen queries in dynamic workloads, respectively, and introduce a hybrid approach that combines their respective strengths by selectively composing them in the process of QPP. We discuss how we can use a training workload to (i) pre-build and materialize such models offline, so that they are readily available for future predictions, and (ii) build new models online as new predictions are needed. All prediction models are built using only static features (available prior to query execution) and the performance values obtained from the offline execution of the training workload.We fully implemented all these techniques and extensions on top of PostgreSQL and evaluated them experimentally by quantifying their effectiveness over analytical workloads, represented by well-established TPC-H data and queries. The results provide quantitative evidence that learning-based modeling for QPP is both feasible and effective for both static and dynamic workload scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning-based Query Performance Modeling and Prediction

Akdere

Çetintemel

Riondato

et al. 2012

2012 IEEE 28th International Conference on Data Engineering

156

196

View full text Add to dashboard Cite

show abstract

“…Comet [12] enables batch processing across streams of data. It is built on top of Dryad [46] and its management relies on its channel concept to transfer a finite set of items over shared memory, TCP pipes or files. However, it is designed to work within the same cluster and does not address the issues of sending continuous stream of events between data-centers.…”

Section: Data Stream Management Systems (Dsms)mentioning

confidence: 99%

JetStream: Enabling high throughput live event streaming on multi-site clouds

Tudoran

Costan

Nano

et al. 2016

Future Generation Computer Systems

View full text Add to dashboard Cite

“…As the cloud model has emerged, this idea of bringing compute to storage has been applied for restricted programming paradigms, e.g., MapReduce [9], or specifically for key-value storage services, e.g., Comet [26]. Also, approaches such as [25] study how to utilize resources in a large cluster by executing data-parallel programs. We propose a more general paradigm that works for every object-based storage repository.…”

Section: Computational Storagementioning

confidence: 99%

A Cloud Environment for Data-intensive Storage Services

Kolodner

Tal

Kyriazis

et al. 2011

2011 IEEE Third International Conference on Cloud Computing Technology and Science

View full text Add to dashboard Cite

Abstract-The emergence of cloud environments has made feasible the delivery of Internet-scale services by addressing a number of challenges such as live migration, fault tolerance and quality of service. However, current approaches do not tackle key issues related to cloud storage, which are of increasing importance given the enormous amount of data being produced in today's rich digital environment (e.g. by smart phones, social networks, sensors, user generated content). In this paper we present the architecture of a scalable and flexible cloud environment addressing the challenge of providing data-intensive storage cloud services through raising the abstraction level of storage, enabling data mobility across providers, allowing computational and content-centric access to storage and deploying new data-oriented mechanisms for QoS and security guarantees. We also demonstrate the added value and effectiveness of the proposed architecture through two real-life application scenarios from the healthcare and media domains.

show abstract

Dryad

Cited by 1,447 publications

References 31 publications

Learning-based Query Performance Modeling and Prediction

Learning-based Query Performance Modeling and Prediction

JetStream: Enabling high throughput live event streaming on multi-site clouds

A Cloud Environment for Data-intensive Storage Services

Contact Info

Product

Resources

About