Directory‐traverse‐cost‐based skew handling for parallel data access

Abstract

SUMMARYTechniques for load balancing by adjusting location of data are widely studied, because they are quite influential in improving the data-access performance and the scalability of a parallel system. To make load balancing effective, distributed directory structures, methods for evaluating loads, and distributed control mechanism for handling skews are important. In this paper, we propose a distributed algorithm to evaluate loads precisely. It counts the loads of intermediate index nodes of a distributed directory, which were ignored in previous publications. We also propose a linear-order distributed skew-handling algorithm adopting the proposed load evaluation algorithm. We validate the effect of our load evaluation algorithm through simulations.

A Compound Parallel Btree for High Scalability and Availability on Chained Declustering Parallel Systems

“…We also assume that P i and B i (i ∈ [1,4]) in the table represent the same amounts of data. We assume that the query frequency is known merely for convenience, because many methods have been proposed for obtaining this information of any PE without any centered nodes, and [37] is one of them which is suitable for the Fat-Btree.…”

Compound Treatment of Chained Declustered Replicas Using a Parallel Btree for High Scalability and Availability

“…For the above example of four nodes, the load can be balanced without data migration if, for example, the query frequency on PE 1 , PE 3 and PE 4 is α, while the query frequency on PE 2 is 2α, as shown in Table 1.We also assume that P i and B i (i ∈ [1,4]) in the table represent the same amounts of data. We assume that the query frequency is known merely for convenience; in practical terms, many methods have already been proposed to find the access probability on any PE, for example, [21] is one of them and is suitable for the Fat-Btree. We first assume the workloads are "all-read".…”