Data placement in Bubba

Abstract

This paper examines the problem of data placement in Bubba, a highly-parallel system for data-intensive applications being developed at MCC. “Highly-parallel” implies that load balancing is a critical performance issue. “Data-intensive” means data is so large that operations should be executed where the data resides. As a result, data placement becomes a critical performance issue. In general, determining the optimal placement of data across processing nodes for performance is a difficult problem. We describe our heuristic approach to solving the data placement problem in Bubba. We then present experimental results using a specific workload to provide insight into the problem. Several researchers have argued the benefits of declustering (i e, spreading each base relation over many nodes). We show that as declustering is increased, load balancing continues to improve. However, for transactions involving complex joins, further declustering reduces throughput because of communications, startup and termination overhead. We argue that data placement, especially declustering, in a highly-parallel system must be considered early in the design, so that mechanisms can be included for supporting variable declustering, for minimizing the most significant overheads associated with large-scale declustering, and for gathering the required statistics.

DYFRAM: dynamic fragmentation and replica management in distributed database systems

“…There are methods that do only fragmentation [1,24,26,33,34] and methods that do only allocation of predefined fragments [3,4,7,10,13,20,30]. Some methods also exist that integrate both tasks [9,11,17,19,25,27,29]. Replication, however, is typically done as a separate task [5,8,15,21,22,32], although some methods, like ours, take an integral view of fragmentation, allocation and replication [11,27,29].…”

“…Our work is on tables and table fragments, but shares the idea of amortizing costs over the expected sequence of operations. In adaptive data placement, the focus has either been on load balancing by data balancing [9,17], or on query analysis [19]. In our algorithms, we seek to place data on the sites where they are being used (by reads or writes), not to balance the load.…”

An Effective Data Placement Strategy for XML Documents

“…Since the complexity of the problem is NP-complete [5], heuristics are normally used to find a nearly optimal solution in a reasonable amount of time. According to the criteria used in reducing costs incurred on resources such as network bandwidth, CPUs, and disks, data placement strategies can be classified into three categories, which are network traffic based [6], size based [7], and access frequency based [8]. The main idea of these approaches is to achieve the minimal load (e.g.…”

Analytische Bestimmung einer Datenallokation für Parallele Data Warehouses

“…Es wurden Studien über solche logischen, i.d.R. 1-dimensionalen Fragmentierungsstrategien durchgeführt, vorwiegend für die SN-Architektur [7,10,17]. Eine solche Strategie erlaubt eine Reduktion des Arbeitsaufwandes lediglich für Anfragen auf dem Fragmentierungsattribut, welches auch eine Komposition mehrerer Attribute sein kann, d.h. der multi-dimensionale und hierarchische Charakter von Sternschemas wird nicht berücksichtigt.…”