Parallelism and its price

Abstract

We describe the use of parallel execution techniques and measure the price of parallel execution in NonStop SQL/MP, a commercial parallel datahase system from Tandem Computers. Non-Stop SQL uses intra-operator parallelism to parallelize joins, groupings and scans. Parallel execution consists of starting up several processes and communicating data between them. Our measurements show (a) Startup costs are negligible when processes are reused rather than created afresh (b) Communication costs are significant -they may exceed the costs of operators such as scan, grouping or join. We also show two counter-examples to the common intuition that parallel execution reduces response time at the expense of increased work-parallel execution may reduce work or may increase response time depending on communication costs.All execution times reported in the paper are scaled. No inferences should be drawn about actual execution times. All query executions reported in the paper were created by bypassing the NonStop SQL optimizer. No inferences should be drawn about the behavior of the optimizer.

Big Data Management in the Cloud: Evolution or Crossroad?

“…Editors of parallel servers offer three main categories of parallel RDBMS based on: (i) shared memory multiprocessor architecture, (ii) shared disks multiprocessor architecture, and (iii) shared nothing multiprocessor architecture (e.g. DBC 1012 Teradata [8,64], Tandem NonStop SQL [23], DB2 Edition [5], ORACLE Parallel Query).…”

“…The main problem to be solved by parallel execution models is to find the best processing-communication trade-off in order to maximize the system throughput and minimize the response time, while maintaining an acceptable cost optimization. In this objective, several efforts have been conducted to reduce inter-processor communication costs, and in particular to avoid the redistribution of data and minimize message transfers [23,30,35].…”

Parallelizing query optimization

“…Yet even though QEPs to execute a query in parallel have been common in commercial products for over a decade [6,15], remarkably there have been no efforts, to the best of our knowledge, to parallelize the query optimization process itself! In the typical shared-nothing or shared-memory multi-node system, a single coordinator node optimizes the query, but many nodes execute it [10].…”