A new approach to modular database systems

Irmert, Florian; Daum, Michael; Meyer-Wegener, Klaus

doi:10.1145/1385486.1385498

Cited by 14 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Operator modularity is a crucial design decision as it significantly affects performance. Determining the right granularity of operators has been a reoccurring topic of research: from the bracket model [23] for parallelization in the early days of databases to objected-oriented modular designs [25], record-oriented components adapted at runtime [37], morsel-driven parallelism [49], and "deep" query optimization [24] more recently. In Modularis, we use the Volcano model [33] as the basis for the interfaces between operators, but we also support collections instead of just flat records.…”

Section: Related Workmentioning

confidence: 99%

“…We show how to design a modular distributed query processing engine with performance comparable to its monolithic counterparts. The topic of modularity in database operators has been visited many times in the past [25,37] and recently [24,47]. However, to our knowledge, we are the first to implement modularity at the hardware platform level, while at the same time formulating concrete design principles.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modularis

Koutsoukos

Müller

Marroquín³

et al. 2021

Proc. VLDB Endow.

View full text Add to dashboard Cite

The enormous quantity of data produced every day together with advances in data analytics has led to a proliferation of data management and analysis systems. Typically, these systems are built around highly specialized monolithic operators optimized for the underlying hardware. While effective in the short term, such an approach makes the operators cumbersome to port and adapt, which is increasingly required due to the speed at which algorithms and hardware evolve. To address this limitation, we present Modularis , an execution layer for data analytics based on sub-operators , i.e., composable building blocks resembling traditional database operators but at a finer granularity. To demonstrate the feasibility and advantages of our approach, we use Modularis to build a distributed query processing system supporting relational queries running on an RDMA cluster, a serverless cloud platform, and a smart storage engine. Modularis requires minimal code changes to execute queries across these three diverse hardware platforms, showing that the sub-operator approach reduces the amount and complexity of the code to maintain. In fact, changes in the platform affect only those sub-operators that depend on the underlying hardware (in our use cases, mainly the sub-operators related to network communication). We show the end-to-end performance of Modularis by comparing it with a framework for SQL processing (Presto), a commercial cluster database (SingleStore), as well as Query-as-a-Service systems (Athena, BigQuery). Modularis outperforms all these systems, proving that the design and architectural advantages of a modular design can be achieved without degrading performance. We also compare Modularis with a hand-optimized implementation of a join for RDMA clusters. We show that Modularis has the advantage of being easily extensible to a wider range of join variants and group by queries, all of which are not supported in the hand-tuned join.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Modularis

Koutsoukos

Müller

Marroquín³

et al. 2021

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…The CoBRA DB project [IDMW08,IFMW08] aims at providing run-time adaptation for DBMS. The focus lies on modularizing a DBMS and supporting module exchange at run-time in a transparent and atomic way.…”

Section: Introductionmentioning

confidence: 99%

A Platform to Support Object Database Research.

Grossniklaus¹,

Leone²,

Spindler³

et al. 2013

JOT

View full text Add to dashboard Cite

Databases play a key role in an increasingly diverse range of applications and settings. New requirements are continually emerging and may differ substantially from one domain to another, sometimes even to the point of conflict. To address these challenges, database systems are evolving to cater for new application domains. Yet little attention has been given to the process of researching and developing database concepts in response to new requirements. We present a platform designed to support database research in terms of experimentation with different aspects of database systems ranging from the data model to the distribution architecture. Our platform is based on the notion of metamodel extension modules, inspired by proposals for adaptive and configurable database management systems. However, rather than building a tailored system from existing components, we focus on the process of designing new components. To qualitatively evaluate our platform, we present a series of case studies where our approach was used successfully to experiment with concepts designed to support a variety of novel application domains.

show abstract

“…The CoBRA DB (Component Based Runtime Adaptable DataBase) project [11] investigates modular DBMSs that can be adapted to different environments by assembling prefabricated CoBRA DB components. The adaptation of the transaction management poses a prominent challenge in this context.…”

Section: Introductionmentioning

confidence: 99%

Semantics of a runtime adaptable transaction manager

Irmert

Lauterwald

Neumann

et al. 2009

Proceedings of the 2009 International Database Engineering &Amp; Applications Symposium on - IDEAS '09

Self Cite

View full text Add to dashboard Cite

Database Management Systems (DBMSs) that can be tailored to specific requirements offer the potential to improve reliability and maintainability and simultaneously the ability to reduce the footprint of the code base. If the requirements of an application change during runtime the DBMS should be adapted without a shutdown. Runtimeadaptation is a new and promising research direction to dynamically change the behavior of a DBMS. Especially the adaptation of the Transaction Manager (TM) states a challenge.In this paper, we present the session semantics of a runtimeadaptable TM. We define preliminaries and assumptions to activate the TM during sessions from a conceptual point of view. The advantages and disadvanteges of different approaches are discussed, especially regarding the occurence of ANSI SQL phenomena. From a technical point of view, we define requirements for the architecture of the TM and the DBMS that arose in our prototype.

show abstract

A new approach to modular database systems

Cited by 14 publications

References 10 publications

Modularis

Modularis

A Platform to Support Object Database Research.

Semantics of a runtime adaptable transaction manager

Contact Info

Product

Resources

About