Fast Multi-Column Sorting in Main-Memory Column-Stores

Xu, Wenjian; Feng, Ziqiang; Lo, Eric

doi:10.1145/2882903.2915205

Cited by 11 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results suggest distinct layouts benefit different operations (e.g., scan for column and aggregation for row). On the other hand, many database operators can be accelerated by applying SIMD, GPU's or FPGA's [35,48,50,57]. However, many accelerated operators require a special data layouts, and thus could benefit from ACCORDA's fast data transformation, enabling more general exploitation of customized formats.…”

Section: Discussion and Related Workmentioning

confidence: 99%

Accelerating raw data analysis with the ACCORDA software and hardware architecture

2019

View full text Add to dashboard Cite

The data science revolution and growing popularity of data lakes make efficient processing of raw data increasingly important. To address this, we propose the ACCelerated Operators for Raw Data Analysis (ACCORDA) architecture. By extending the operator interface (subtype with encoding) and employing a uniform runtime worker model, ACCORDA integrates data transformation acceleration seamlessly, enabling a new class of encoding optimizations and robust high-performance raw data processing. Together, these key features preserve the software system architecture, empowering state-of-art heuristic optimizations to drive flexible data encoding for performance. ACCORDA derives performance from its software architecture, but depends critically on the acceleration of the Unstructured Data Processor (UDP) that is integrated into the memory-hierarchy, and accelerates data transformation tasks by 16x-21x (parsing, decompression) to as much as 160x (deserialization) compared to an x86 core. We evaluate ACCORDA using TPC-H queries on tabular data formats, exercising raw data properties such as parsing and data conversion. The ACCORDA system achieves 2.9x-13.2x speedups when compared to SparkSQL, reducing raw data processing overhead to a geomean of 1.2x (20%). In doing so, ACCORDA robustly matches or outperforms prior systems that depend on caching loaded data, while computing on raw, unloaded data. This performance benefit is robust across format complexity, query predicates, and selectivity (data statistics). ACCORDA's encoding-extended operator interface unlocks aggressive encoding-oriented optimizations that deliver 80% average performance increase over the 7 affected TPC-H queries.

show abstract

Section: Discussion and Related Workmentioning

confidence: 99%

Accelerating raw data analysis with the ACCORDA software and hardware architecture

2019

View full text Add to dashboard Cite

show abstract

“…Cao et al [9] present query optimization techniques for queries with multiple window functions (e.g., reusing existing partitioning and ordering properties), which are also applicable and indeed are directly enabled by our approach. Except for Xu et al [37], much work on optimizing sort algorithms for modern hardware [10,14] has focused on small tuple sizes. Grouping sets have been proposed by Gray et al [19] in 1997, and consequently there have been many proposals for optimizing the grouping order: Phan and Michiardi's [30] fairly recent paper offers a good overview.…”

Section: Related Workmentioning

confidence: 99%

Building Advanced SQL Analytics From Low-Level Plan Operators

Kohn

Leis

Neumann

2021

Proceedings of the 2021 International Conference on Management of Data

View full text Add to dashboard Cite

Analytical queries virtually always involve aggregation and statistics. SQL offers a wide range of functionalities to summarize data such as associative aggregates, distinct aggregates, ordered-set aggregates, grouping sets, and window functions. In this work, we propose a unified framework for advanced statistics that composes all flavors of complex SQL aggregates from low-level plan operators. These operators can reuse materialized intermediate results, which decouples monolithic aggregation logic and speeds up complex multi-expression queries. The contribution is therefore twofold: our framework modularizes aggregate implementations, and outperforms traditional systems whenever multiple aggregates are combined. We integrated our approach into the high-performance database system Umbra and experimentally show that we compute complex aggregates faster than the state-of-the-art HyPer system.

show abstract

“…Lastly, we try to understand the quality of our estimation function. Since our primary goal is not the estimation accuracy as discussed, we evaluate the rank [49] of our estimation function instead. Specifically, there is a perfect ranking of 100 system settings obtained in the baseline experiments, where the one with the best completion time, i.e., Best, is rank 1st, and Worst, has rank 100-th.…”

Section: Estimation Qualitymentioning

confidence: 99%

“…In contrast, if X 2 turns out to rank 100-th in the oracle, that means the estimated "optimal" setting turns out is the worst one among the 100 settings. The notion of rank based on an actual oracle has been used in [49] and it was shown that it is way more informative than using the notion of error when evaluating the quality of an estimation function. So now, for each segment (iterations 1 to 60 is segment 1, iterations 61 to 120 is segment 2, etc.…”

Section: Estimation Qualitymentioning

confidence: 99%

Towards Self-Tuning Parameter Servers

Liu¹

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years, many applications have been driven advances by the use of Machine Learning (ML). Nowadays, it is common to see industrial-strength machine learning jobs that involve millions of model parameters, terabytes of training data, and weeks of training. Good efficiency, i.e., fast completion time of running a specific ML job, therefore, is a key feature of a successful ML system. While the completion time of a longrunning ML job is determined by the time required to reach model convergence, practically that is also largely influenced by the values of various system settings. In this paper, we contribute techniques towards building self-tuning parameter servers. Parameter Server (PS) is a popular system architecture for large-scale machine learning systems; and by self-tuning we mean while a long-running ML job is iteratively training the expert-suggested model, the system is also iteratively learning which system setting is more efficient for that job and applies it online. While our techniques are general enough to various PSstyle ML systems, we have prototyped our techniques on top of TensorFlow. Experiments show that our techniques can reduce the completion times of a variety of long-running TensorFlow jobs from 1.4× to 18×.

show abstract

Fast Multi-Column Sorting in Main-Memory Column-Stores

Cited by 11 publications

References 30 publications

Accelerating raw data analysis with the ACCORDA software and hardware architecture

Accelerating raw data analysis with the ACCORDA software and hardware architecture

Building Advanced SQL Analytics From Low-Level Plan Operators

Towards Self-Tuning Parameter Servers

Contact Info

Product

Resources

About