Data placement in Bubba

Abstract

This paper examines the problem of data placement in Bubba, a highly-parallel system for data-intensive applications being developed at MCC. "Highly-parallel" implies that load balancing is a critical performance issue. "Data-intensive" means data is so large that operationsshould be executed where the data resides. As a result, data placement becomes a critical performance issue.In general, determining the optimal placement of data across processing nodes for performance is a difficult problem. We describe our heuristic approach to solving the data placement problem in Bubba. We then present experimental results using a specific workload to provide insight into the problem. Several researchers have argued the benefits of declustering (i.e., spreading each base relation over many nodes). We show that as declustering is increased, load balancing continues to improve. However, for transactions involving complex joins, further declustering reduces throughput because of communications, startup and termination overhead.We argue that data placement, especially declustering, in a highly-parallel system must be considered early in the design, so that mechanisms can be included for supporting variable declustering, for minimizing the most significant overheads associated with large-scale declustering, and for gathering the required statistics.

Big Data Management in the Cloud: Evolution or Crossroad?

“…In a parallel RDBMS, it becomes possible to: (i) improve I/O bandwidth by fully exploiting the parallelism of read operations of one or more relations (ii) apply data locality principle (operators are performed where/or very close to the data are located), and (iii) facilitate load balancing to maximize throughput. The key problem with data partitioning, also called data placement, consists in reaching and holding the best tradeoff between processing and communication [17]. Two approaches make it possible to solve the data placement problem of a set of relations in a parallel RDBMS.…”

Hauptspeicherdatenbanken für Unternehmensanwendungen

“…In der Regel wird hierbei eine horizontale Partitionierung der Daten durchgeführt, um diese auf unterschiedliche Systeme verteilen zu können. Um eine Verteilung der einzelnen Tupel zu ermöglichen, kann wie von Copeland et al in [8] beschrieben eine Verteilung nach Heat erfolgen, d.h. wie oft einzelne Tupel verwendet werden. Eine Herausforderung hierbei ist es bei Verringern oder Vergrößern der Hardwareressourcen die vorhandenen Tupel auf neue Server zu verteilen oder auf weniger Server zu konsolidieren.…”

Browsing and placement of multi-resolution images on parallel disks

“…A standard technique for improving disk performance is to control the placement of data on disks. Several data placement techniques have been used to overcome the I/O bottleneck of secondary storage [2][3][4]21,9,13]. Some studies [12,17] have relied on the well understood data structure and access patterns of relational databases to develop placement techniques.…”