Profiling dataflow systems on multiple abstraction levels

Beischl, Alexander; Kersten, Timo; Bandle, Maximilian; Giceva, Jana; Neumann, Thomas

doi:10.1145/3447786.3456254

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 The query set is available online. 5 To measure the quality of the estimates, we report the qerror. The q-error measures the factor that an estimate differs from the ground truth.…”

Section: Isolating the Impact Of Aggregate Estimatesmentioning

confidence: 99%

“…We found that for static estimation, we get the least error with the 4 https://github.com/cwida/public_bi_benchmark. 5 https://db.in.tum.de/~fent/data/aggEst.tgz. Fig.…”

Section: Isolating the Impact Of Aggregate Estimatesmentioning

confidence: 99%

“…The execution of this query spends most time in the groupjoin. Figure 28 shows a trace of the operator activity over the execution [5]. Note that to get a precise trace, we gathered it on a recent Ice Lake system and limited parallelism, so the total execution time is not comparable to our previous evaluation.…”

Section: Tpc-h Q3mentioning

confidence: 99%

See 2 more Smart Citations

Practical planning and execution of groupjoin and nested aggregates

2022

View full text Add to dashboard Cite

Groupjoins combine execution of a join and a subsequent group-by. They are common in analytical queries and occur in about "Equation missing" of the queries in TPC-H and TPC-DS. While they were originally invented to improve performance, efficient parallel execution of groupjoins can be limited by contention in many-core systems. Efficient implementations of groupjoins are highly desirable, as groupjoins are not only used to fuse group-by and join, but are also useful to efficiently execute nested aggregates. For these, the query optimizer needs to reason over the result of aggregation to optimally schedule it. Traditional systems quickly reach their limits of selectivity and cardinality estimations over computed columns and often treat group-by as an optimization barrier. In this paper, we present techniques to efficiently estimate, plan, and execute groupjoins and nested aggregates. We propose four novel techniques, aggregate estimates to predict the result distributions of aggregates, parallel groupjoin execution for scalable execution of groupjoins, index groupjoins, and a greedy eager aggregation optimization technique that introduces nested preaggregations to significantly improve execution plans. The resulting system has improved estimates, better execution plans, and a contention-free evaluation of groupjoins, which speeds up TPC-H and TPC-DS queries significantly.

show abstract

“…4 The query set is available online. 5 To measure the quality of the estimates, we report the qerror. The q-error measures the factor that an estimate differs from the ground truth.…”

Section: Isolating the Impact Of Aggregate Estimatesmentioning

confidence: 99%

“…We found that for static estimation, we get the least error with the 4 https://github.com/cwida/public_bi_benchmark. 5 https://db.in.tum.de/~fent/data/aggEst.tgz. Fig.…”

Section: Isolating the Impact Of Aggregate Estimatesmentioning

confidence: 99%

See 1 more Smart Citation

Practical planning and execution of groupjoin and nested aggregates

2022

View full text Add to dashboard Cite

show abstract

“…Additionally, to materialize the economic benefits of serverless computing, systems have to be serverless themselves. Unfortunately, current distributed tracing systems (e. g., Dapper [16]), database observability approaches (e. g., [7]), or serverless application tracing (e. g., [9,12]) do not cover these challenges.…”

Section: Introductionmentioning

confidence: 99%

Doppler

Bodner

Pietz

Bollmeier

et al. 2022

Proceedings of the International Workshop on Big Data in Emergent Distributed Environments

View full text Add to dashboard Cite

Analyzing and understanding query execution dynamics in distributed cloud-based databases is difficult and requires laborious system designs. Serverless query execution, with its massive amount of small, short-lived, and stateless query workers, is even more challenging. To meet this challenge and materialize the economic benefits of serverless computing, all system components have to be serverless themselves. We demonstrate Doppler, a serverless toolkit designed to trace serverless data processing systems with minimal performance and cost overhead and to provide a deep understanding of their query execution. We highlight Doppler's features and capabilities through a proof-of-concept implementation with the serverless data processing system Skyrise. CCS CONCEPTS• Information systems → Database utilities and tools; Database query processing; Relational parallel and distributed DBMSs; • Computer systems organization → Cloud computing.

show abstract