In-memory BLU acceleration in IBM's DB2 and dashDB: Optimized for modern workloads and hardware architectures

Barber, Ronald; Lohman, Guy M.; Raman, V.; Sidle, Richard; Lightstone, Sam; Schiefer, Berni

doi:10.1109/icde.2015.7113372

Cited by 18 publications

(11 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there has been a surge in interests in recent years [222]. Examples of commercial in-memory relational databases include SAP HANA [77], VoltDB [150], Oracle TimesTen [38], SolidDB [40], IBM DB2 with BLU Acceleration [223], [224], Microsoft Hekaton [37], NuoDB [43], eXtremeDB [225], Pivotal SQLFire [226], and MemSQL [42]. There are also well known research/open-source projects such as HStore [36], HyPer [35], Silo [39], Crescando [227], HYRISE [176], and MySQL Cluster NDB [228].…”

Section: In-memory Relational Databasesmentioning

confidence: 99%

In-Memory Big Data Management and Processing: A Survey

Zhang

Chen

Ooi

et al. 2015

IEEE Trans. Knowl. Data Eng.

356

156

View full text Add to dashboard Cite

Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.

show abstract

Section: In-memory Relational Databasesmentioning

confidence: 99%

In-Memory Big Data Management and Processing: A Survey

Zhang

Chen

Ooi

et al. 2015

IEEE Trans. Knowl. Data Eng.

356

156

View full text Add to dashboard Cite

show abstract

“…The work most closely related to Data Blocks is the storage format of IBM DB2 BLU [4,30], which also consists of blocks representing all columns of a sequence of tuples stored in compressed columnar format inside the block (a concept introduced as PAX [3]) and provides mechanisms for early evaluation of range filters inside the scan. The main differences are that HyPer avoids bit-packing to keep positional access fast and introduces the PSMAs that further improve the evaluation of early selections.…”

Section: Related Workmentioning

confidence: 99%

Data Blocks

Lang

Mühlbauer

Funke³

et al. 2016

Proceedings of the 2016 International Conference on Management of Data

102

View full text Add to dashboard Cite

This work aims at reducing the main-memory footprint in high performance hybrid OLTP & OLAP databases, while retaining high query performance and transactional throughput. For this purpose, an innovative compressed columnar storage format for cold data, called Data Block s is introduced. Data Blocks further incorporate a new lightweight index structure called Positional SMA that narrows scan ranges within Data Blocks even if the entire block cannot be ruled out. To achieve highest OLTP performance, the compression schemes of Data Blocks are very lightweight , such that OLTP transactions can still quickly access individual tuples. This sets our storage scheme apart from those used in specialized analytical databases where data must usually be bit-unpacked. Up to now, high-performance analytical systems use either vectorized query execution or "just-in-time" (JIT) query compilation. The fine-grained adaptivity of Data Blocks necessitates the integration of the best features of each approach by an interpreted vectorized scan subsystem feeding into JIT-compiled query pipelines. Experimental evaluation of HyPer, our full-fledged hybrid OLTP & OLAP database system, shows that Data Blocks accelerate performance on a variety of query workloads while retaining high transaction throughput.

show abstract

“…Yet, in many cases, the large number of concurrent user requests and the increasingly complex queries they issue force storing data on disk. Hence, modern data management systems need to be able to use different levels of the storage hierarchy interchangeably [3].…”

Section: Deep Storage Hierarchiesmentioning

confidence: 99%

Databases and hardware

Ailamaki¹

2015

Proc. VLDB Endow.

View full text Add to dashboard Cite

Fast query and transaction processing is the goal of 40 years of database research and the reason of existence for many new database system architectures. In data management, system performance means acceptable response time and throughput on critical-path operations, ideally with scalability guarantees. Performance is improved with top-of-the line research on data processing algorithms; efficiency, however, is contingent on seamless collaboration between the database software and hardware and storage devices. In 1980, the goal was to minimize disk accesses; in 2000, memory replaced disks in terms of access costs. Nowadays performance is synonymous to scalability; scalability, in turn, translates into sustainable and predictable use of hardware resources in the face of embarrassing parallelism and deep storage hierarchies while minimizing energy needs-a challenging goal in multiple dimensions. We discuss work done in the past four decades to tighten the interaction between the database software and underlying hardware and show that, as application and microarchitecture roadmaps evolve, the effort of maintaining smooth collaboration blossoms into a multitude of interesting research avenues with direct technological impact.

show abstract

In-memory BLU acceleration in IBM's DB2 and dashDB: Optimized for modern workloads and hardware architectures

Cited by 18 publications

References 6 publications

In-Memory Big Data Management and Processing: A Survey

In-Memory Big Data Management and Processing: A Survey

Data Blocks

Databases and hardware

Contact Info

Product

Resources

About