DB2 with BLU acceleration

Raman, V.; Attaluri, Gopi; Barber, Ronald; Chainani, Naresh; Kalmuk, David; KulandaiSamy, Vincent; Leenstra, J.; Lightstone, Sam; Liu, Shaorong; Lohman, Guy M.; Malkemus, Timothy; Mueller, Rene; Pandis, Ippokratis; Schiefer, Berni; Sharpe, D R; Sidle, Richard; Storm, Adam; Zhang, Liping

doi:10.14778/2536222.2536233

Cited by 191 publications

(19 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the HTAP design space that we study, we consider two engines, one OLTP and one OLAP, which can be either logically HyPer-MVOCC [32], MemSQL, IBM BLU [36] MVCC OLAP (version traversal) SAP HANA [17] Delta-Versioning OLAP (version traversal), OLTP (record chains)…”

Section: Data Freshnessmentioning

confidence: 99%

Adaptive HTAP through Elastic Resource Scheduling

Raza

Chrysogelos

Anadiotis

et al. 2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

Modern Hybrid Transactional/Analytical Processing (HTAP) systems use an integrated data processing engine that performs analytics on fresh data, which are ingested from a transactional engine. HTAP systems typically consider data freshness at design time, and are optimized for a fixed range of freshness requirements, addressed at a performance cost for either OLTP or OLAP. The data freshness and the performance requirements of both engines, however, may vary with the workload. We approach HTAP as a scheduling problem, addressed at runtime through elastic resource management. We model an HTAP system as a set of three individual engines: an OLTP, an OLAP and a Resource and Data Exchange (RDE) engine. We devise a scheduling algorithm which traverses the HTAP design spectrum through elastic resource management, to meet the data freshness requirements of the workload. We propose an in-memory system design which is non-intrusive to the current state-of-art OLTP and OLAP engines, and we use it to evaluate the performance of our approach. Our evaluation shows that the performance benefit of our system for OLAP queries increases over time, reaching up to 50% compared to static schedules for 100 query sequences, while maintaining a small, and controlled, drop in the OLTP throughput. CCS CONCEPTS • Information systems → DBMS engine architectures; Main memory engines; Database transaction processing; Online analytical processing engines.

show abstract

Section: Data Freshnessmentioning

confidence: 99%

Adaptive HTAP through Elastic Resource Scheduling

Raza

Chrysogelos

Anadiotis

et al. 2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…For instance, the in-memory DBMS SAP HANA [8] uses horizontal and vertical layouts for On-line Transaction Processing (OLTP) and On-line Analytical Processing (OLAP) workloads, respectively. In a similar way, in DB2 [17] horizontal and vertical layouts can be used for the same table-space. However, these layouts are fixed and non-modifiable at runtime.…”

Section: Related Workmentioning

confidence: 99%

ResilientStore: A Heuristic-Based Data Format Selector for Intermediate Results

Munir

Romero

Abelló

et al. 2016

Model and Data Engineering

View full text Add to dashboard Cite

Abstract. Large-scale data analysis is an important activity in many organizations that typically requires the deployment of data-intensive workflows. As data is processed these workflows generate large intermediate results, which are typically pipelined from one operator to the following. However, if materialized, these results become reusable, hence, subsequent workflows need not recompute them. There are already many solutions that materialize intermediate results but all of them assume a fixed data format. A fixed format, however, may not be the optimal one for every situation. For example, it is well-known that different data fragmentation strategies (e.g., horizontal and vertical) behave better or worse according to the access patterns of the subsequent operations. In this paper, we present ResilientStore, which assists on selecting the most appropriate data format for materializing intermediate results. Given a workflow and a set of materialization points, it uses rule-based heuristics to choose the best storage data format based on subsequent access patterns. We have implemented ResilientStore for HDFS and three different data formats: SequenceFile, Parquet and Avro. Experimental results show that our solution gives 18% better performance than any solution based on a single fixed format.

show abstract

“…This demonstration presents Wildfire, which seeks to support this new class of scalable HTAP applications that require ultra-high data ingest rates in excess of 1 million inserts per second per node while concurrently performing analytics queries at speeds comparable to the latest Relational DBMS (RDBMS) engines [6,7,8,9] , about one half order of magnitude faster than existing Spark SQL [4] . We leverage the Spark infrastructure to provide a massively parallel and elastic HTAP solution that enables different types of analytics via the Spark ecosystem (Spark SQL, MLLib, GraphX, etc), but can also ingest data at very high rates.…”

Section: Sigmodmentioning

confidence: 99%

“…Wildfire uses a columnar query engine that has many similarities to the query engine of DB2 with BLU Acceleration [9] . Wildfire processes composite queries, such as joins, in multiple stages.…”

Section: Wildfire Engine: Storage and Processingmentioning

confidence: 99%

“…However, recent advances in speeding up analytics queries [6,7,8,9] now make it possible to combine these systems into one integrated system called Hybrid Transactional and Analytics Processing (HTAP). Doing so permits analytics to see the latest transactions, reducing the business risk of making wrong decisions, such as fraud detection, on stale data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Wildfire

Barber

Huras

Lohman

et al. 2016

Proceedings of the 2016 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

We demonstrate Hybrid Transactional and Analytics Processing (HTAP) on the Spark platform by the Wildfire prototype, which can ingest up to ≈6 million inserts per second per node and simultaneously perform complex SQL analytics queries. Here, a simplified mobile application uses Wildfire to recommend advertising to mobile customers based upon their distance from stores and their interest in products sold by these stores, while continuously graphing analytics results as those customers move and respond to the ads with purchases.

show abstract

DB2 with BLU acceleration

Cited by 191 publications

References 12 publications

Adaptive HTAP through Elastic Resource Scheduling

Adaptive HTAP through Elastic Resource Scheduling

ResilientStore: A Heuristic-Based Data Format Selector for Intermediate Results

Wildfire

Contact Info

Product

Resources

About