Dynamic Data Grid Replication Strategy Based on Internet Hierarchy

Abstract

Abstract. In data grid, large quantity of data files is produced and data replication is applied to reduce data access time. Efficiently utilizing grid resources becomes important research issue since available resources in grid are limited while large number of workloads and large size of data files are produced. Dynamic replication in data grid aims to reduce data access time and to utilize network and storage resources efficiently. This paper proposes a novel dynamic replication strategy, called BHR, which reduces data access time by avoiding network congestions in a data grid network. With BHR strategy, we can take benefits from 'network-level locality' which represents that required file is located in the site which has broad bandwidth to the site of job execution. We evaluate BHR strategy by implementing it in an OptorSim, a data grid simulator initially developed by European Data Grid Projects. The simulation results show that BHR strategy can outperform other optimization techniques in terms of data access time when hierarchy of bandwidth appears in Internet. BHR extends current site-level replica optimization study to the network-level.

A New Architecture for Group Replication in Data Grid

“…Also a dynamic replica optimization strategy (HRS) is suggested where the nearby data has a higher priority to access than to generate new replicas. The simulation results showed that HCS successfully reduced data access time and the amount of intercluster-communications in comparison with LRU (Least Recently Used), LFU (Least Frequently Used), BHR (Bandwidth Hierarchy-based Replication) [10]. Since the file transmission time is the most important factor to influence the job execution time for data-intensive jobs in data grids, HCS with HRS can reduce the file transmission time effectively by virtue of valid scheduling and proper data replication.…”

Dynamic Replication Based on Availability and Popularity in the Presence of Failures

“…Usually, dynamic replication is triggered when the popularity of a file exceeds a certain threshold [13]. The authors of [15,18] use dynamic replication to conserve bandwidth in a hierarchical environment. The work in [9] studies the replication dynamics in different topologies such as ring, tree, and hybrid.…”

“…To solve the problem of data loss in both cases (unpopularity or failure), we proposed, with some improvements, an approach of dynamic replication that combines replication that is based on the availability of the work proposed in [11,15] and replication based on the popularity of the work given in [8,16]. In this approach the minimum number of replicas for each piece of data is the number of replicas that meets the availability desired by the administrator.…”

Two Level Job Scheduling and Data Replication in Data Grid

“…Since every domain knows average number of access for each file and then deletes those files whose access is less than the average, and replicates other files in its local storage. In [4], an algorithm for a 2-level hierarchical structure based on internet hierarchy (BHR) has been introduced which only considers dynamic replication and does not consider scheduling. Nodes in the first level are connected to each other with high speed networks and in the second level via internet.…”

“…In large-scale data-intensive applications, data transfer time is the primary cause of job execution delay. To mitigate the effect of such problems and facilitate efficient system performance, replication strategies and algorithms are employed [4]. The replication mechanism determines which file should be replicated, when to create new replicas and where the new replicas should be placed.…”