Query Centric Partitioning and Allocation for Partially Replicated Database Systems

Towards learning a partitioning advisor with deep reinforcement learning

“…Different from these approaches our approach implements a dedicated online phase that is able to cope with inaccuracies of the cost model. Another approach [21] optimizes both analytical and transactional workloads by partially allocating already partitioned tables in an optimal manner to minimize runtime or maximize throughput. Different from this approach, which is only focusing on the allocation, in this paper we provide a new solution to find a partitioning scheme which is an orthogonal problem to data allocation.…”

“…Deciding for complex schemata with many tables and possible join paths which tables should be copartitioned is a non-trivial task since this not only depends on the database schemata but also other factors such as table sizes, the query workload (i.e., which joins are actually important and how often tables are joined), or hardware characteristics such as network speed. Some techniques have already been proposed to find good data partitionings for analytical workloads over distributed databases [16,21]. However, the existing approaches either rely on heuristics only [21] or solely use cost estimates from the query optimizer [16], which often do not reflect the real execution costs for more complex queries.…”

“…Some techniques have already been proposed to find good data partitionings for analytical workloads over distributed databases [16,21]. However, the existing approaches either rely on heuristics only [21] or solely use cost estimates from the query optimizer [16], which often do not reflect the real execution costs for more complex queries.…”

Ephemeral Partially Replicated Databases

“…Since these databases are isolated, all the data elements necessary to satisfy a query are required to be available on a single database. Designing a partially replicated database system is usually a two-step process involving partitioning and allocation [11].…”

“…Nevertheless, the compute scalability achieved using the fully replicated databases doesn't grow linearly with the number of database replications. Each new replication of the data requires additional copying of data from one database to another, and thereby significant amount of resources are wasted on reading and writing data [11]. Furthermore, while the fully replicated database system helps achieve a high degree of database availability and compute scalability, it fails to provide storage scalability.…”