Tuning a parallel database algorithm on a shared‐memory multiprocessor

Abstract

Database query processing can benefit significantly from parallelism. Parallel database algorithms combine substantial CPU and YO activity, memory requirements, and massive data exchange between processes, all of which must be considered to obtain optimal performance. Since parallel external sorting is a very typical example, we have focused on sorting to tune Volcano, a new query processing system.The purpose of the Volcano project is to provide efficient, extensible tools for query and request processing in novel application domains, particularly in object-oriented and scientific database systems, and for experimental database performance research. It includes all query processing algorithms conventionally used in relational database systems as well as several new ones, and can execute all of them in parallel. In this article, we present Volcano's parallel external sorting algorithm and a sequence of enhancements to improve its performance. We obtained very good absolute performance, 84 seconds for 100 MB of data, as well as near-linear speedup with sixteen CPUs and disks. Furthermore, these results were achieved on a shared-memory machine despite the common belief that parallel query processing is best implemented on distributed-memory systems. We detail our tuning measures and report on their effectiveness.

Multi-level transaction management for complex objects: Implementation, performance, parallelism

“…Rather the performance difference is because strategy S2/PS saves calls to the buffer manager as it defers the writing of after-images. This reduces some CPU overhead, and decreases the contention on internal latches that are used to synchronize the access to the buffer manager's frame control blocks (Graefe and Thakkar, 1992). Such latch contention is also the major reason for the drop of performance that both $2 and S2/PS suffer when the DMP exceeds 12 (i.e., the number of processors).…”

Alphasort: A cache-sensitive parallel external sort

“…Sort's priceperformance metric normalizes variations in software and hardware configuration. The basic idea is to compute the 5-year cost of the hardware and software, and then prorate that cost for the elapsed time of the sort [1,12]. A one minute sort on a machine with a 5-year cost of a million dollars would cost 38 cents (0.38$).…”

“…DebitCredit: a market basket of database reads and writes, terminal IO, and transaction commits to measure on-line transaction processing performance (OLTP). This benchmark evolved to become the TPC-A transactions-per-second and dollars-pertransaction-per-second metrics [12]. Scan: copy a thousand 100-byte records from disk-to-disk with transaction protection.…”

AlphaSort

“…DebitCredit: a market basket of database reads and writes, terminal IO, and transaction commits to measure online transaction processing performance (OLTP). This benchmark evolved to become the TPC-A transactionsper-second and dollars-per-transaction-per-second metrics [12]. Sean: copy a thousand 100-byte records from disk-todisk with transaction protection.…”