Dynamic Query Scheduling in Parallel Data Warehouses

Abstract

Abstract. Data warehouse queries pose challenging performance problems that often necessitate the use of parallel database systems (PDBS). Although dynamic load balancing is of key importance in PDBS, to our knowledge it has not yet been investigated thoroughly for parallel data warehouses. In this study, we propose a scheduling strategy that simultaneously considers both processors and disks while utilizing the load balancing potential of a Shared Disk architecture. We compare the performance of this new method to several other approaches in a comprehensive simulation study, incorporating skew aspects and typical data warehouse features such as star schemas.

Single-scan: a fast star-join query processing algorithm

“…As the multi‐core processors have seen a very fast evolution in the last decade and the scalability of distributed systems has also improved significantly , one can assume that the most efficient way to lower the query processing time is to parallelize their execution. Numerous studies have been performed regarding the parallel execution of queries in a database or data warehouse . Hadoop‐DB is a hybrid data warehouse environment that uses several relational database management system (PostgreSql) as data nodes and Hadoop + Hive as the execution engine.…”

A Query Beehive Algorithm for Data Warehouse Buffer Management and Query Scheduling

“…This interaction is common in Multiple Query Optimization (MQO) problem and physical design in RDW (Sellis, 1988). The MQO is related to other optimization problems such as Buffer Management (BMP) (Cornell & Yu, 1989) and Query Scheduling (QSP) (Chipara, Lu & Roman, 2007;Märtens, Rahm & Stöhr, 2002), and the joint problem of BMP and QSP (Gupta, Sudarshan & Viswanathan, 2001;Tan & Lu, 1995;Gupta, Sudarshan & Viswanathan, 2001). These problems are studied in three main levels: off-line, dynamic (adaptive) and on-line.…”

“…The QSP has also been studied in several environments: centralized (Thomas, Diwan & Sudarshan, 2006), distributed and parallel databases (Märtens, Rahm & Stöhr, 2002). It has been proved as strongly NP-complete problem (Thomas, Diwan & Sudarshan, 2006).…”