No False Negatives: Accepting All Useful Schedules in a Fast Serializable Many-Core System

Abstract

Concurrency control is one of the most performance critical steps in modern many-core database systems. Achieving higher throughput on multi-socket servers is difficult and many concurrency control algorithms reduce the amount of accepted schedules in favor of transaction throughput or relax the isolation level which introduces unwanted anomalies. Both approaches lead to unexpected transaction behavior that is difficult to understand by the database users. We introduce a novel multi-version concurrency protocol that achieves high performance while reducing the number of aborted schedules to a minimum and providing the best isolation level. Our approach leverages the idea of a graph-based scheduler that uses the concept of conflict graphs. As conflict serializable histories can be represented by acyclic conflict graphs, our scheduler maintains the conflict graph and allows all transactions that keep the graph acyclic. All conflict serializable schedules can be accepted by such a graph-based algorithm due to the conflict graph theorem. Hence, only transaction schedules that truly violate the serializability constraints need to abort. Our developed approach is able to accept the useful intersection of commit order preserving conflict serializable (COCSR) and recoverable (RC) schedules which are the two most desirable classes in terms of correctness and user experience. We show experimentally that our graph-based scheduler has very competitive throughput in pure transactional workloads while providing fewer aborts and improved user experience. Our multi-version extension helps to efficiently perform long-running read transactions on the same up-to-date database. Moreover, our graph-based scheduler can outperform the competitors on mixed workloads.

On the Impact of Memory Allocation on High-Performance Query Processing

“…Some online transaction processing (OLTP) systems try to reduce the allocation overhead by managing their allocated memory in chunks to increase performance for small transactional queries [4,22,23]. However, most database systems process both transactional and analytical queries.…”

“…In-memory hash joins and aggregations can be implemented in many different ways which can influence the allocation pattern heavily [2,3,15,24]. Some online transaction processing (OLTP) systems try to reduce the allocation overhead by managing their allocated memory in chunks to increase performance for small transactional queries [4,22,23]. However, most database systems process both transactional and analytical queries.…”

An Analysis of Concurrency Control Protocols for In-Memory Databases with CCBench (Extended Version)

“…Whereas previous studies mostly evaluated scalability and did not explore the behavior of protocols when thread parallelism was set to a high degree [30,32,33,35,37,39,43,44,46,56,63,65,69,70,73,74,76,77], we fixed the thread parallelism at 224 and analyzed protocols for various settings. We classified a variety of methods on the basis of three performance factors: cache, delay, and version lifetime.…”

“…Cavalia [4] Healing [73] Cavalia [4] HTCC [74] Cavalia [4] Cicada [46] Cicada [6] ACC [63] Doppel [53] Latch-free SSN [68] ERMIA [9] FOEDUS [44] FOEDUS [10] MOCC [69] FOEDUS [10] Silo [65] Silo [14] IC3 [70] Silo [14] STOv2 [39] STO [15] CormCC [62] Original TicToc [76] NP DBx1000 [7] Strife [56] DBx1000 [7] AOCC [37](β) DBx1000 [7] EWV [35] Original BCC [77] Silo [14] Batching [32] φ Cicada [6] SGT [33] Original [13] Doppel [53] Original…”

“…These modern protocols use a variety of optimization methods, and their behaviors depend on the workload characteristics (e.g., thread count, skew, read ratio, cardinality, payload size, transaction size, and read-modify-write or not). Recent new proposal studies have compared these protocols with conventional ones [30,32,33,35,37,39,43,44,46,56,63,65,69,70,73,74,76,77], and recent analytical studies have compared the performance of conventional protocols on a common platform [23,24,72,75]. These studies mostly evaluated protocol scalability.…”