The Design and Implementation of Modern Column-Oriented Database Systems

Abstract

In this article, we survey recent research on column-oriented database systems, or column-stores, where each attribute of a table is stored in a separate file or region on storage. Such databases have seen a resurgence in recent years with a rise in interest in analytic queries that perform scans and aggregates over large portions of a few columns of a table. The main advantage of a column-store is that it can access just the columns needed to answer such queries. We specifically focus on three influential research prototypes, MonetDB [46], VectorWise [18], and C-Store [88]. These systems have formed the basis for several wellknown commercial column-store implementations. We describe their similarities and di erences and discuss their specific architectural features for compression, late materialization, join processing, vectorization and adaptive indexing (database cracking).

A new approximate query engine based on intelligent capture and fast transformations of granulated data summaries

“…(Unless this is the case of a nested query, where the result of subquery can remain in a summarized form, like we did with rough queries in Kowalski et al 2013.) This phase could be referred to as materialization, though it should not be confused with a standard meaning of materialization in columnar databases (Abadi et al 2013). Alternatively, if the knowledge capture layer is regarded as responsible for aforementioned information granulation (Zadeh 1997), then translation of query result summaries into final approximate results can be treated as information degranulation.…”

“…In that earlier framework, packrows were described by simple summaries accessible independently from the underlying data. It combined the ideas taken from other database technologies (Abadi et al 2013) and the theory of rough sets (Pawlak and Skowron 2007), by means of using summaries to classify data packs as relevant, irrelevant and partially relevant for particular SELECT statements -by analogy to deriving rough set positive, negative and boundary regions of the considered concepts, respectively. Such higher-level classifications were useful to limit the amounts of compressed data packs required to access to finish calculations.…”

“…For the considered engine, it would be a step toward the paradigms of multi-level granular data analytics (Yao 2016). Yet another aspect refers to vertical data organization (Abadi et al 2013), which -in our case -means an independent access to collections of histograms, special values, gaps, cooccurrence ratios, etc. This way, for every query, we can grasp these components of stored data summaries that are required to execute particular operations.…”

The Column-oriented Data Store Performance Considerations

“…This trend has been called the not only SQL or NoSQL and was one of the outcomes of a rise of interactive, especially social, web services within the web 2.0 movement. [6] The most significant developments in the area of columnar data stores are the C-Store [2] [7] and MonetDB [7].…”

The column-oriented database partitioning optimization based on the natural computing algorithms

“…Nevertheless, it needs to be stated explicitly that both models are not related. Modern column-oriented data stores have been covered in [2]. Implementation details of the Apache Cassandra columnfamily store have been described in [3].…”