Towards a framework for orchestrated distributed database evaluation in the cloud

Seybold, Daniel

doi:10.1145/3152688.3152693

Cited by 5 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we derive a set of open evaluation challenges, which will be addressed in future work: (i) evaluating additional operational models implementations; (ii) evaluating additional storage backends; (iii) consolidation of containerized DBMS and processing applications on top of VMs and (iv) the feasibility of DBMS containerization and orchestration for different DBMS. These challenges will be addressed within [9].…”

Section: Discussionmentioning

confidence: 99%

The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS

Seybold¹,

Hauser²,

Eisenhart³

et al. 2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Containers emerged as cloud resource offerings. While the advantages of containers, such as easing the application deployment, orchestration and adaptation, work well for stateless applications, the feasibility of containerization of stateful applications, such as database management system (DBMS), still remains unclear due to potential performance overhead. The myriad of container operation models and storage backends even raises the complexity of operating a containerized DBMS. Here, we present an extensible evaluation methodology to identify performance overhead of a containerized DBMS by combining three operational models and two storage backends. For each combination a memory-bound and disk-bound workload is applied. The results show a clear performance overhead for containerized DBMS on top of virtual machines (VMs) compared to physical resources. Further, a containerized DBMS on top of VMs with different storage backends results in a tolerable performance overhead. Building upon these baseline results, we derive a set of open evaluation challenges for containerized DBMSs.

show abstract

Section: Discussionmentioning

confidence: 99%

The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS

Seybold¹,

Hauser²,

Eisenhart³

et al. 2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Secondly, we identify requirements for enabling holistic DBMS elasticity evaluations. to DBMS workload tools, this process supports the operatorside by managing cloud resources and the DBMS cluster [18], [19]. In addition, the process handles workload issuing and client-side evaluation metrics.…”

Section: Dbms Elasticity Evaluation Challengesmentioning

confidence: 99%

Kaa: Evaluating Elasticity of Cloud-Hosted DBMS

Seybold¹,

Volpert²,

Wesner³

et al. 2019

2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)

Self Cite

View full text Add to dashboard Cite

Auto-scaling is able to change the scale of an application at runtime. Understanding the application characteristics, scaling impact as well as the workload, an auto-scaler aligns the acquired resources to match the current workload. For distributed Database Management Systems (DBMS) forming the backend of many large-scale cloud applications, it is currently an open question to what extent they support scaling at run-time. In particular, elasticity properties of existing distributed DBMS are widely unknown and difficult to evaluate and compare. This paper presents a comprehensive methodology for the evaluation of the elasticity of distributed DBMS. On the basis of this methodology, we introduce a framework that automates the full evaluation process. We validate the framework by defining significant elasticity scenarios for a case study that comprises two DBMS for write-heavy and read-heavy workloads of different intensities. The results show that scalable distributed DBMS are not necessarily elastic and that adding more instances to a cluster at run-time may even decrease the experienced performance.

show abstract

“…In the following, we present the multi-domain evaluation framework Mowgli 2 that builds upon existing DBMS evaluation concepts [34]. Mowgli automates the entire evaluation process shown in Figure 2 by enabling the definition and execution of portable and reproducible evaluation scenarios via a loosely coupled and [5] and combines them with an extensible DBMS catalogue, an auto-generated cloud resource catalogue and a workload catalogue.…”

Section: Mowglimentioning

confidence: 99%

Mowgli

Seybold

Keppler

Gründler

et al. 2019

Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

Big Data and IoT applications require highly-scalable database management system (DBMS), preferably operated in the cloud to ensure scalability also on the resource level. As the number of existing distributed DBMS is extensive, the selection and operation of a distributed DBMS in the cloud is a challenging task. While DBMS benchmarking is a supportive approach, existing frameworks do not cope with the runtime constraints of distributed DBMS and the volatility of cloud environments. Hence, DBMS evaluation frameworks need to consider DBMS runtime and cloud resource constraints to enable portable and reproducible results. In this paper we present Mowgli, a novel evaluation framework that enables the evaluation of non-functional DBMS features in correlation with DBMS runtime and cloud resource constraints. Mowgli fully automates the execution of cloud and DBMS agnostic evaluation scenarios, including DBMS cluster adaptations. The evaluation of Mowgli is based on two IoT-driven scenarios, comprising the DBMSs Apache Cassandra and Couchbase, nine DBMS runtime configurations, two cloud providers with two different storage backends. Mowgli automates the execution of the resulting 102 evaluation scenarios, verifying its support for portable and reproducible DBMS evaluations. The results provide extensive insights into the DBMS scalability and the impact of different cloud resources. The significance of the results is validated by the correlation with existing DBMS evaluation results.Since the era of RDBMS, their selection is guided by domain-specific benchmarks that have evolved together with distributed DBMSs.

show abstract

Towards a framework for orchestrated distributed database evaluation in the cloud

Cited by 5 publications

References 10 publications

The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS

The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS

Kaa: Evaluating Elasticity of Cloud-Hosted DBMS

Mowgli

Contact Info

Product

Resources

About