Squall

Abstract

For data-intensive applications with many concurrent users, modern distributed main memory database management systems (DBMS) provide the necessary scale-out support beyond what is possible with single-node systems. These DBMSs are optimized for the short-lived transactions that are common in on-line transaction processing (OLTP) workloads. One way that they achieve this is to partition the database into disjoint subsets and use a single-threaded transaction manager per partition that executes transactions one-ata-time in serial order. This minimizes the overhead of concurrency control mechanisms, but requires careful partitioning to limit distributed transactions that span multiple partitions. Previous methods used off-line analysis to determine how to partition data, but the dynamic nature of these applications means that they are prone to hotspots. In these situations, the DBMS needs to reconfigure how data is partitioned in real-time to maintain performance objectives. Bringing the system off-line to reorganize the database is unacceptable for on-line applications.To overcome this problem, we introduce the Squall technique for supporting live reconfiguration in partitioned, main memory DBMSs. Squall supports fine-grained repartitioning of databases in the presence of distributed transactions, high throughput client workloads, and replicated data. An evaluation of our approach on a distributed DBMS shows that Squall can reconfigure a database with no downtime and minimal overhead on transaction latency.

Zeus

“…However, their datastore designs that support re-sharding and provide strong consistency operate at a sub-Mtps throughput. For instance, Squall [23] and Rococo [47] report up to 100 Ktps per server, and Rocksteady [37] up to 700 Ktps per server.…”

High availability in cheap distributed key value storage

“…Get (ke y) Once the backup has determined that it has sent all of its local records to the recovery primary, it notifies the recovery primary via a BatchPull response with 0 entries. Similarly to tracking the progress of migrating tuples in Squall [14], the backup determines when all local records have been successfully transferred to the recovery primary. The recovery primary can then transition into steady state operation as a normal primary.…”

“…Rocksteady and Squall (H-store) Rocksteady [21] and Squall [14] are both systems that enable reconfiguration of inmemory datastores. Both focus on reconfiguring in a way that minimizes the performance impact during the reconfiguration process.…”

Gloss

“…Distributed database migration Recent work [17,20,21,42,47] focuses on live OLTP workload migration in distributed databases for load balancing and elastic scaling.…”

“…Albatross [17] migrates active OLTP workloads of a sharedstorage DBMS by iteratively copying the database cache between nodes. Squall [20] migrates data tuples between nodes of a distributed in-memory database while maintaining ACID semantics. Squall maintains responsiveness by first migrating tuples being accessed by transactions.…”