Replication at the speed of change

Butterstein, Dennis; Martín, Daniel; Stolze, Knut; Beier, Felix; Jia, Zhenhong; Wang, Lingyun

doi:10.14778/3415478.3415548

Cited by 7 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, due to the flexibility of ETL, it is more likely suitable for big-data analysis (e.g., MapReduce and Spark). Some previous studies [34,126] indeed blur the boundary between real-time ETL and HTAP. In our survey, we differentiate them because they have evolved into two different research directions and have different optimization focuses in recent years.…”

Section: Survey Organizationmentioning

confidence: 83%

See 1 more Smart Citation

A survey on hybrid transactional and analytical processing

Song,

Zhou,

Cui

et al. 2024

The VLDB Journal

View full text Add to dashboard Cite

To provide applications with the ability to analyze fresh data and eliminate the time-consuming ETL workflow, hybrid transactional and analytical (HTAP) systems have been developed to serve online transaction processing and online analytical processing workloads in a single system. In recent years, HTAP systems have attracted considerable interest from both academia and industry. Several new architectures and technologies have been proposed. This paper provides a comprehensive overview of these HTAP systems. We review recently published papers and technical reports in this field and broadly classify existing HTAP systems into two categories based on their data formats: monolithic and hybrid HTAP. We further classify hybrid HTAP into four sub-categories based on their storage architecture: row-oriented, column-oriented, separated, and hybrid. Based on such a taxonomy, we outline each stream’s design challenges and performance issues (e.g., the contradictory format demand for monolithic HTAP). We then discuss potential solutions and their trade-offs by reviewing noteworthy research findings. Finally, we summarize emerging HTAP applications, benchmarks, future trends, and open problems.

show abstract

Section: Survey Organizationmentioning

confidence: 83%

“…For instance, several works [164,166] exploit the advantages of task parallelization to speed up the workflow. Other works [34] use batch processing to improve the throughput of transformation. As both HTAP and real-time ETL strive to provide high-performance OLTP and OLAP with fresh data, they share a similar design target.…”

Section: Survey Organizationmentioning

confidence: 99%

A survey on hybrid transactional and analytical processing

Song,

Zhou,

Cui

et al. 2024

The VLDB Journal

View full text Add to dashboard Cite

show abstract

“…Relational DBMSs (RDBMS) have traditionally been put forward as general-purpose systems that can be deployed to handle different types of workloads. Nonetheless, seminal studies (Michael Stonebraker et al, 2007;Michael Stonebraker & Cetintemel, 2005) as well as recent literature (Butterstein et al, 2020;Rompf & Amin, 2019) support the notion that systems that allow the configuration of the DBMS to target a specific type of workload perform better.…”

Section: Introductionmentioning

confidence: 99%

HTAP With Reactive Streaming ETL

Camilleri

Vella

Nezval

2021

Journal of Cases on Information Technology

View full text Add to dashboard Cite

In database management systems (DBMSs), query workloads can be classified as online transactional processing (OLTP) or online analytical processing (OLAP). These often run within separate DBMSs. In hybrid transactional and analytical processing (HTAP), both workloads may execute within the same DBMS. This article shows that it is possible to run separate OLTP and OLAP DBMSs, and still support timely business decisions from analytical queries running off fresh transactional data. Several setups to manage OLTP and OLAP workloads are analysed. Then, benchmarks on two industry standard DBMSs empirically show that, under an OLTP workload, a row-store DBMS sustains a 1000 times higher throughput than a columnar DBMS, whilst OLAP queries are more than 4 times faster on a columnar DBMS. Finally, a reactive streaming ETL pipeline is implemented which connects these two DBMSs. Separate benchmarks show that OLTP events can be streamed to an OLAP database within a few seconds.

show abstract

“…These tasks rarely happen within database systems but in external tools [37,54] requiring the data to be extracted from database systems [30]. Thus, current research mostly focuses on eliminating the extraction process [26,41,8,56,51] and developing systems that combine data management and machine learning [35]. In contrast, in this paper, we argue that code generation allows database systems to perform well for machine learning when training neural networks [55] based on matrix algebra in SQL only [27,38,32,48,47].…”

Section: Introductionmentioning

confidence: 99%

Versioning in Main-Memory Database Systems

Schüle

Karnowski

Schmeißer

et al. 2019

Proceedings of the 31st International Conference on Scientific and Statistical Database Management

View full text Add to dashboard Cite

Although database systems perform well in data access and manipulation, their relational model hinders data scientists from formulating machine learning algorithms in SQL. Nevertheless, we argue that modern database systems perform well for machine learning algorithms expressed in relational algebra. To overcome the barrier of the relational model, this paper shows how to transform data into a relational representation for training neural networks in SQL: We first describe building blocks for data transformation, model training and inference in SQL-92 and their counterparts using an extended array data type. Then, we compare the implementation for model training and inference using array data types to the one using a relational representation in SQL-92 only. The evaluation in terms of runtime and memory consumption proves the suitability of modern database systems for matrix algebra, although specialised array data types perform better than matrices in relational representation.

show abstract

Replication at the speed of change

Cited by 7 publications

References 12 publications

A survey on hybrid transactional and analytical processing

A survey on hybrid transactional and analytical processing

HTAP With Reactive Streaming ETL

Versioning in Main-Memory Database Systems

Contact Info

Product

Resources

About