Optimizing queries over partitioned tables in MPP systems

Antova, Lyublena; El-Helw, Amr; Soliman, Mohamed A.; Gu, Zhongxian; Petropoulos, Michalis; Waas, Florian

doi:10.1145/2588555.2595640

Cited by 17 publications

(19 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dependencies between operators in the logical expression are captured as references between groups. For example, InnerJoin [1,2] refers to Group 1 and Group 2 as children. Optimization takes place as described in the following steps.…”

Section: Optimization Workflowmentioning

confidence: 99%

“…Transformation rules that generate logically equivalent expressions are triggered. For example, a Join Commutativity rule is triggered to generate InnerJoin [2,1] out of InnerJoin [1,2]. Exploration results in adding new group expressions to existing groups and possibly creating new groups.…”

Section: Optimization Workflowmentioning

confidence: 99%

“…Figure 7 shows the optimization of req. #1 by InnerHashJoin [1,2]. For this request, one of the alternative plans is aligning child distributions based on join condition, so that tuples to be joined are co-located 2 .…”

Section: Optimization Workflowmentioning

confidence: 99%

“…The corresponding child request in the local hash table of Sort is req #4. The best group expression for req #4 is InnerHashJoin [1,2]. We thus link Sort to InnerHashJoin.…”

Section: Optimization Workflowmentioning

confidence: 99%

See 3 more Smart Citations

Orca

Soliman

Antova

Raghavan

et al. 2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

The performance of analytical query processing in data management systems depends primarily on the capabilities of the system's query optimizer. Increased data volumes and heightened interest in processing complex analytical queries have prompted Pivotal to build a new query optimizer.In this paper we present the architecture of Orca, the new query optimizer for all Pivotal data management products, including Pivotal Greenplum Database and Pivotal HAWQ. Orca is a comprehensive development uniting state-of-theart query optimization technology with own original research resulting in a modular and portable optimizer architecture.In addition to describing the overall architecture, we highlight several unique features and present performance comparisons against other systems.

show abstract

Section: Optimization Workflowmentioning

confidence: 99%

Section: Optimization Workflowmentioning

confidence: 99%

Section: Optimization Workflowmentioning

confidence: 99%

“…The corresponding child request in the local hash table of Sort is req #4. The best group expression for req #4 is InnerHashJoin [1,2]. We thus link Sort to InnerHashJoin.…”

Section: Optimization Workflowmentioning

confidence: 99%

See 2 more Smart Citations

Orca

Soliman

Antova

Raghavan

et al. 2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

show abstract

“…The difficulty is that, with the increase in the amount of data, how to carry out these areas of inquiry and storage. Massively Parallel Processing (MPP) architecture can address these challenges through distributed storage and querying across multiple nodes and processes [4,5]. MPP is usually used in mathematical modeling of heavy computation, arduous database processing, complex weather modeling and other fields, which is characterized by accommodating multiple processing servers running in parallel.…”

Section: Introductionmentioning

confidence: 99%

Data Dissemination and Parallel Processing Techniques Research Based on Massively Parallel Processing

Sun¹,

Deng²,

Nie³

et al. 2017

dtcse

View full text Add to dashboard Cite

Abstract. In the current distributed database system architecture enterprise-class, the massively parallel processing architecture is used frequently. This method can be used to carry out large-scale analysis of data through distributed across multiple nodes and storage and query process, from its scope of application produce simple reports to perform complex analytics workloads. However, due to the characteristics of shared-nothing MPP technology, to carry out large-scale data analysis query and maintain data consistency there are some difficulties. In this paper, a relational SQL-based query parsing distributed MPP data distribution and parallel processing technology, the goal is to maintain and improve the consistency of distributed data query speed. First SQL query analysis section, according to the syntax analysis, semantic analysis and sentence parsing steps such order; in the form of work distribution node/data node in the data distribution phase, all tasks emanating from the work of a distribution node, all need to treated results are returned to the node; when parallel processing, each node needs to store a copy of the lookup table, and on each node concurrent execution of SQL statements for each query. Experimental results show that the proposed MPP data distribution and parallel processing scheme can support large volume of data processing, ensuring data consistency in the premise of improving query processing speed.

show abstract

Big Data Query Engines

Soliman¹

2017

Handbook of Big Data Technologies

View full text Add to dashboard Cite

Optimizing queries over partitioned tables in MPP systems

Cited by 17 publications

References 4 publications

Orca

Orca

Data Dissemination and Parallel Processing Techniques Research Based on Massively Parallel Processing

Big Data Query Engines

Contact Info

Product

Resources

About