Native store extension for SAP HANA

Abstract

We present an overview of SAP HANA's Native Store Extension (NSE). This extension substantially increases database capacity, allowing to scale far beyond available system memory. NSE is based on a hybrid in-memory and paged column store architecture composed from data access primitives. These primitives enable the processing of hybrid columns using the same algorithms optimized for traditional HANA's in-memory columns. Using only three key primitives, we fabricated byte-compatible counterparts for complex memory resident data structures (e.g. dictionary and hash-index), compressed schemes (e.g. sparse and run-length encoding), and exotic data types (e.g. geo-spatial). We developed a new buffer cache which optimizes the management of paged resources by smart strategies sensitive to page type and access patterns. The buffer cache integrates with HANA's new execution engine that issues pipelined prefetch requests to improve disk access patterns. A novel load unit configuration, along with a unified persistence format, allows the hybrid column store to dynamically switch between inmemory and paged data access to balance performance and storage economy according to application demands while reducing Total Cost of Ownership (TCO). A new partitioning scheme supports load unit specification at table, partition, and column level. Finally, a new advisor recommends optimal load unit configurations. Our experiments illustrate the performance and memory footprint improvements on typical customer scenarios.

Growth of relational model: Interdependence and complementary to big data

“…Storage tiering [85], or transferring data between different storage types to suit data I/O characteristics to that of storage, is a strategy that administrators have been using for many years. Data aging is the easiest and most common reason for this.…”

Testing Very Large Database Management Systems: The Case of SAP HANA

“…Considering the given examples, software testing can provide information for architects to initiate activities for improvements. A high number of bugs can trigger a further investigation of the types of the bugs and based on the types of bugs can trigger a re-implementation to avoid such bugs in the future [78]. A high number of bugs that cannot be reproduced can indicate complexity that is not wellunderstood.…”