A comparison of knives for bread slicing

Jindal, Alekh; Palatinus, Endre; Pavlov, Vladimir; Dittrich, Jens

doi:10.14778/2536336.2536338

Cited by 23 publications

(24 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The decision which attributes to cover is similar to computing the optimal vertical partitioning for a table. Various applicable algorithms are presented in [12]. Figure 6(b) shows the performance of covered cracking over different numbers of projected attributes.…”

Section: Improving Tuple Reconstructionmentioning

confidence: 99%

The uncracked pieces in database cracking

Schuhknecht

Jindal²,

Dittrich

2013

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

Database cracking has been an area of active research in recent years. The core idea of database cracking is to create indexes adaptively and incrementally as a side-product of query processing. Several works have proposed different cracking techniques for different aspects including updates, tuple-reconstruction, convergence, concurrency-control, and robustness. However, there is a lack of any comparative study of these different methods by an independent group. In this paper, we conduct an experimental study on database cracking. Our goal is to critically review several aspects, identify the potential, and propose promising directions in database cracking. With this study, we hope to expand the scope of database cracking and possibly leverage cracking in database engines other than MonetDB.We repeat several prior database cracking works including the core cracking algorithms as well as three other works on convergence (hybrid cracking), tuple-reconstruction (sideways cracking), and robustness (stochastic cracking) respectively. We evaluate these works and show possible directions to do even better. We further test cracking under a variety of experimental settings, including high selectivity queries, low selectivity queries, and multiple query access patterns. Finally, we compare cracking against different sorting algorithms as well as against different main-memory optimised indexes, including the recently proposed Adaptive Radix Tree (ART). Our results show that: (i) the previously proposed cracking algorithms are repeatable, (ii) there is still enough room to significantly improve the previously proposed cracking algorithms, (iii) cracking depends heavily on query selectivity, (iv) cracking needs to catch up with modern indexing trends, and (v) different indexing algorithms have different indexing signatures.

show abstract

Section: Improving Tuple Reconstructionmentioning

confidence: 99%

The uncracked pieces in database cracking

Schuhknecht

Jindal²,

Dittrich

2013

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The key idea is to develop an optimization model to satisfy one or more criteria to improve the I/O performance of databases. Partitioning of databases, similar to physical design problems has been proven to be NP-Hard due to the estimation errors in both system and workload parameters [53][54][55], therefore extensive work has been done by the database community [56][57][58][59][60][61][62][63][64].…”

Section: Multi-tier Storagementioning

confidence: 99%

“…Dynamically adaptive variations of these concepts have been explored thoroughly in the design of modern datastores [53,54,61]. These methods are used in online data partitioning such as O2P, H20 [60], etc., and disk based analytical databases [54,63,65]. In the Big Data ecosystem, most prominently the concepts of [56][57][58][59] have laid strong footing for the data layout design on HDFS [54,65,66].…”

Section: Multi-tier Storagementioning

confidence: 99%

Host managed contention avoidance storage solutions for Big Data

Mishra

Somani

2017

J Big Data

View full text Add to dashboard Cite

The performance gap between compute and storage is fairly considerable. This results in a mismatch between the application needs from storage and what storage can deliver. The full potential of storage devices cannot be harnessed till all layers of I/O hierarchy function efficiently. Despite advanced optimizations applied across various layers along the odyssey of data access, the I/O stack still remains volatile. The problems associated due to the inefficiencies in data management get amplified in Big Data shared resource environments. The Linux OS (host) block layer is the most critical part of the I/O hierarchy, as it orchestrates the I/O requests from different applications to the underlying storage. Unfortunately, despite it's significance, the block layer, essentially the block I/O scheduler, hasn't evolved to meet the needs of Big Data. We have designed and developed two contention avoidance storage solutions, collectively known as "BID: Bulk I/O Dispatch" in the Linux block layer specifically to suit multitenant, multi-tasking shared Big Data environments. Hard disk drives (HDDs) form the backbone of data center storage. The data access time in HDDs is majorly governed by disk arm movements, which usually occurs when data is not accessed sequentially. Big Data applications exhibit evident sequentiality but due to the contentions amongst other I/O submitting applications, the I/O accesses get multiplexed which leads to higher disk arm movements. BID schemes aim to exploit the inherent I/O sequentiality of Big Data applications to improve the overall I/O completion time by reducing the avoidable disk arm movements. In the first part, we propose a dynamically adaptable block I/O scheduling scheme BID-HDD for disk based storage. BID-HDD tries to recreate the sequentiality in I/O access in order to provide performance isolation to each I/O submitting process. Through trace driven simulation based experiments with cloud emulating MapReduce benchmarks, we show the effectiveness of BID-HDD which results in 28-52% lesser time for all I/O requests than the best performing Linux disk schedulers. In the second part, we propose a hybrid scheme BID-Hybrid to exploit SCM's (SSDs) superior random performance to further avoid contentions at disk based storage. BID-Hybrid is able to efficiently offload non-bulky interruptions from HDD request queue to SSD queue using BID-HDD for disk request processing and multi-q FIFO architecture for SSD. This results in performance gain of 6-23% for MapReduce workloads when compared to BID-HDD and 33-54% over best performing Linux scheduling scheme. BID schemes as a whole is aimed to avoid contentions for disk based storage I/Os following system constraints without compromising SLAs.

show abstract

“…Column grouping is an important database technique and has been extensively studied (e.g., [14,25,30,35,37,46]). While many column grouping approaches exist, the general principle is to put into the same group the columns that are frequently queried together in the workload [25] and adopt a row-major layout within each column group.…”

Section: Motivationmentioning

confidence: 99%

Skipping-oriented partitioning for columnar layouts

et al. 2016

View full text Add to dashboard Cite

As data volumes continue to grow, modern database systems increasingly rely on data skipping mechanisms to improve performance by avoiding access to irrelevant data. Recent work [39] proposed a fine-grained partitioning scheme that was shown to improve the opportunities for data skipping in row-oriented systems. Modern analytics and big data systems increasingly adopt columnar storage schemes, and in such systems, a row-based approach misses important opportunities for further improving data skipping. The flexibility of column-oriented organizations, however, comes with the additional cost of tuple reconstruction. In this paper, we develop Generalized Skipping-Oriented Partitioning (GSOP), a novel hybrid data skipping framework that takes into account these rowbased and column-based tradeoffs. In contrast to previous columnoriented physical design work, GSOP considers the tradeoffs between horizontal data skipping and vertical partitioning jointly. Our experiments using two public benchmarks and a real-world workload show that GSOP can significantly reduce the amount of data scanned and improve end-to-end query response times over the state-of-the-art techniques.

show abstract

A comparison of knives for bread slicing

Cited by 23 publications

References 11 publications

The uncracked pieces in database cracking

The uncracked pieces in database cracking

Host managed contention avoidance storage solutions for Big Data

Skipping-oriented partitioning for columnar layouts

Contact Info

Product

Resources

About