Scalability analysis of declustering methods for multidimensional range queries

Abstract

Efficient storage and retrieval of multiattribute data sets has become one of the essential requirements for many data-intensive applications. The Cartesian product file has been known as an effective multiattribute file structure for partial-match and best-match queries. Several heuristic methods have been developed to decluster Cartesian product files across multiple disks to obtain high performance for disk accesses. Although the scalability of the declustering methods becomes increasingly important for systems equipped with a large number of disks, no analytic studies have been done so far. In this paper, we derive formulas describing the scalability of two popular declustering methods¦Disk Modulo and Fieldwise Xor¦for range queries, which are the most common type of queries. These formulas disclose the limited scalability of the declustering methods, and this is corroborated by extensive simulation experiments. From the practical point of view, the formulas given in this paper provide a simple measure that can be used to predict the response time of a given range query and to guide the selection of a declustering method under various conditions.

Servicing range queries on multidimensional datasets with partial replicas

“…Several research projects have looked at improving I/O performance using different declustering techniques [9,14]. Parallel file systems and I/O libraries have also been a widely studied research topic, and many such systems and libraries have been developed [3,7,13].…”

“…Chunks in each replica that intersect the query are categorized as partial or full chunks and into different fragments, and the respective goodness values of the fragments are calculated (steps 2-6). For a given query Q, let us denote the set of all fragments as F and the list of all chosen fragments in decreasing order of goodness value as S. We can apply our greedy search over F (the while loop over steps [8][9][10][11][12][13][14][15][16][17][18]. We choose the fragment with the largest goodness value, move it from F to S, and modify Q by subtracting the range contained by this fragment.…”

“…Efficient access to data also depends on how well the data has been distributed across multiple storage nodes. The goal of declustering [9,14] is to distribute the data across as many storage units as possible so that data elements that satisfy a query can be retrieved from many sources in parallel. Caching is yet another optimization that targets multiple query workloads [1,10,19,21].…”

A Parallel Implementation of 4-Dimensional Haralick Texture Analysis for Disk-Resident Image Datasets

“…In addition, distributing the input dataset across multiple storage nodes has the advantage that data retrieval can be parallelized. A number of techniques have been developed for partitioning and declustering multi-dimensional datasets [15,16,31,33]. Obviously, the effectiveness of a particular distribution depends on how well it matches the common data access and query patterns of the target application class.…”

Optimizing retrieval and processing of multi-dimensional scientific datasets

“…Since data is accessed through range queries, it is desirable to have data items that are close to each other in the multi-dimensional space placed in the same chunk. Chunks are distributed across the disks attached to ADR back-end nodes using a declustering algorithm [10,16] to achieve I/O parallelism during query processing. Each chunk is assigned to a single disk, and is read and/or written during query processing only by the local processor to which the disk is attached.…”