Column-oriented database systems

Abstract

Column-oriented database systems (column-stores) have attracted a lot of attention in the past few years. Column-stores, in a nutshell, store each database table column separately, with attribute values belonging to the same column stored contiguously, compressed, and densely packed, as opposed to traditional database systems that store entire records (rows) one after the other. Reading a subset of a table's columns becomes faster, at the potential expense of excessive disk-head seeking from column to column for scattered reads or updates. After several dozens of research papers and at least a dozen of new column-store start-ups, several questions remain. Are these a new breed of systems or simply old wine in new bottles? How easily can a major row-based system achieve column-store performance? Are column-stores the answer to effortlessly support large-scale data-intensive applications? What are the new, exciting system research problems to tackle? What are the new applications that can be potentially enabled by column-stores? In this tutorial, we present an overview of column-oriented database system technology and address these and other related questions.

Fast scans on key-value stores

“…Locality is important for both disk-based and in-memory scans. Specifically, adding scans to the feature list creates the following locality conflicts: Scan vs. Get/Put Most analytical systems use a columnar storage layout to increase locality [2]. KVS, in contrast, typically favor a row-oriented layout in order to process get/put requests without the need to materialize records [4].…”

“…Column-major vertically partitions the data and stores a whole column of a table (or set of records) as a contiguous sequence of bytes. Such a column-major layout is beneficial for scans as analytical queries often involve only a subset of the columns [25,3,2,45,9,8]. In addition, columnmajor supports vector operations (SIMD) to further speed up bulk operations and scans on modern hardware [47,49].…”

Design of Efficient Index Management for Column-based Big Databases

“…To improve read efficiency, columnar values are typically densely packed, and using light-weight compression scheme. Especially, columnoriented database facilitates vast improvements in compression, which can result in an additional reduction in storage access while maintaining high performance [4].…”

An Effective and Efficient MapReduce Algorithm for Computing BFS-Based Traversals of Large-Scale RDF Graphs

“…As regards the proper data storage solution, Hadoop's ecosystem incorporates HBase [29]. HBase is a distributed column-oriented database (e.g., [30]), which also falls in the category of NoSQL storage systems (e.g., [31]), that founds on the underlying Hadoop Distributed File System (HDFS) (e.g., [32]) for common data processing routines. HBase has been proved to be highly scalable.…”