A distributed load balancing algorithm for deduplicated storage

Abstract

While deduplication brings the advantage of significant space savings in storage, it nevertheless incurs the overhead of maintaining huge metadata. Updating such huge metadata during the data migration that arises due to load balancing activity results in significant overhead. In order to reduce this metadata update overhead, this paper proposes a suitable alternate index that tracks the data blocks even when they migrate across the nodes without explicitly storing the location information. In addition, a virtual server-based load balancing (VSLB) algorithm has been proposed in order to reduce the migration overhead. The experimental results indicate that the proposed index reduces the metadata update overhead by 74% when compared to the existing index. Furthermore, VSLB reduces the migration overhead by 33% when compared to the existing ID reassignment-based load balancing approach.