Scaling Up Mixed Workloads: A Battle of Data Freshness, Flexibility, and Scheduling

Psaroudakis, Iraklis; Wolf, Florian; May, Norman; Neumann, Thomas; Böhm, Alexander; Ailamaki, Anastasia; Sattler, Kai-Uwe

doi:10.1007/978-3-319-15350-6_7

Cited by 15 publications

(19 citation statements)

References 12 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Task-based execution is used in numerous modern DBMS and research prototypes, including SAP HANA [18], IBM DB2 BLU [20], HyPer [16], etc. Task scheduling works for both OLTP and OLAP workloads, although we have shown that workload management, e.g., through task prioritization, is essential to avoid the OLAP workload dominating over a concurrent OLTP workload [19]. The main reason is that analytical workloads are typically more aggressively parallelized than transactional workloads.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Extending database task schedulers for multi-threaded application code

Wolf

Psaroudakis²,

May

et al. 2015

Proceedings of the 27th International Conference on Scientific and Statistical Database Management

Self Cite

View full text Add to dashboard Cite

Modern databases can run application logic defined in stored procedures inside the database server to improve application speed. The SQL standard specifies how to call external stored routines implemented in programming languages, such as C, C++, or JAVA, to complement declarative SQLbased application logic. This is beneficial for scientific and analytical algorithms because they are usually too complex to be implemented entirely in SQL. At the same time, database applications like matrix calculations or data mining algorithms benefit from multi-threading to parallelize computeintensive operations. Multi-threaded application code, however, introduces a resource competition between the threads of applications and the threads of the database task scheduler. In this paper, we show that multi-threaded application code can render the database's workload scheduling ineffective and decrease the core throughput of the database by up to 50%. We present a general approach to address this issue by integrating shared memory programming solutions into the task schedulers of databases. In particular, we describe the integration of OpenMP into databases. We implement and evaluate our approach using SAP HANA. Our experiments show that our integration does not introduce overhead, and can improve the throughput of core database operations by up to 15%.

show abstract

Section: Related Workmentioning

confidence: 99%

“…We run this algorithm with the three input sizes shown in Table 2. Initially, keys and values in the input data structures are filled with random integers between [0, 99], and [0,19] respectively. The grouping is hash-based, using a C++11 std::unordered_map.…”

Section: Data Set Matrix Sizementioning

confidence: 99%

Extending database task schedulers for multi-threaded application code

Wolf

Psaroudakis²,

May

et al. 2015

Proceedings of the 27th International Conference on Scientific and Statistical Database Management

Self Cite

View full text Add to dashboard Cite

show abstract

“…Operations are encapsulated in tasks, and a pool of worker threads is used to process them. In our previous work, we showed how task scheduling can be used in a NUMA-agnostic main-memory column-store to efficiently process mixed OLTP and OLAP workloads [28,29]. We showed how stealing and a flexible concurrency level can help to saturate CPU resources without oversubscribing them, and how a concurrency hint can be used to adjust the task granularity of analytical partitionable operations to avoid unnecessary scheduling overhead.…”

Section: Related Workmentioning

confidence: 99%

Scaling up concurrent main-memory column-store scans

et al. 2015

Self Cite

View full text Add to dashboard Cite

Main-memory column-stores are called to efficiently use modern non-uniform memory access (NUMA) architectures to service concurrent clients on big data. The efficient usage of NUMA architectures depends on the data placement and scheduling strategy of the column-store. Most column-stores choose a static strategy that involves partitioning all data across the NUMA architecture, and employing a stealingbased task scheduler. In this paper, we implement different strategies for data placement and task scheduling for the case of concurrent scans. We compare these strategies with an extensive sensitivity analysis. Our most significant findings include that unnecessary partitioning can hurt throughput by up to 70%, and that stealing memory-intensive tasks can hurt throughput by up to 58%. Based on our analysis, we envision a design that adapts the data placement and task scheduling strategy to the workload.

show abstract

“…Modern database systems need to support mixed workloads of online transaction processing (OLTP) and online analytical processing (OLAP) workloads [11,17,18]. OLTP workloads contain short-lived, light transactions which read or update small portions of data, while OLAP workloads contain long-running, heavy transactions which reads large portions of data.…”

Section: Introductionmentioning

confidence: 99%

“…However, this approach requires an extremely expensive hardware. Previous work such as Hyper [11,18] focuses on scaling up mixed workloads in a single hardware host, which eventually limits the maximum scalability of analytical query processing.…”

Section: Introductionmentioning

confidence: 99%

Parallel replication across formats in SAP HANA for scaling out mixed OLTP/OLAP workloads

Lee¹,

Moon

Kim³

et al. 2017

Proc. VLDB Endow.

View full text Add to dashboard Cite

Modern in-memory database systems are facing the need of efficiently supporting mixed workloads of OLTP and OLAP. A conventional approach to this requirement is to rely on ETL-style, application-driven data replication between two very different OLTP and OLAP systems, sacrificing realtime reporting on operational data. An alternative approach is to run OLTP and OLAP workloads in a single machine, which eventually limits the maximum scalability of OLAP query performance. In order to tackle this challenging problem, we propose a novel database replication architecture called Asynchronous Parallel Table Replication (ATR). ATR supports OLTP workloads in one primary machine, while it supports heavy OLAP workloads in replicas. Here, rowstore formats can be used for OLTP transactions at the primary, while column-store formats are used for OLAP analytical queries at the replicas. ATR is designed to support elastic scalability of OLAP query performance while it minimizes the overhead for transaction processing at the primary and minimizes CPU consumption for replayed transactions at the replicas. ATR employs a novel optimistic lock-free parallel log replay scheme which exploits characteristics of multi-version concurrency control (MVCC) in order to enable real-time reporting by minimizing the propagation delay between the primary and replicas. Through extensive experiments with a concrete implementation available in a commercial database system, we demonstrate that ATR achieves sub-second visibility delay even for updateintensive workloads, providing scalable OLAP performance without notable overhead to the primary.

show abstract

Scaling Up Mixed Workloads: A Battle of Data Freshness, Flexibility, and Scheduling

Cited by 15 publications

References 12 publications

Extending database task schedulers for multi-threaded application code

Extending database task schedulers for multi-threaded application code

Scaling up concurrent main-memory column-store scans

Parallel replication across formats in SAP HANA for scaling out mixed OLTP/OLAP workloads

Contact Info

Product

Resources

About