Data Access Frequency based Data Replication Method using Erasure Codes in Cloud Storage System

Abstract

Cloud storage system uses a distributed file system for storing and managing data. Traditional distributed file system makes a triplication of data in order to restore data loss in disk failure. However, enforcing data replication method increases storage utilization and causes extra I/O operations during replication process. In this paper, we propose a data replication method using erasure codes in cloud storage system to improve storage space efficiency and I/O performance. In particular, according to data access frequency, the proposed method can reduce the number of data replications but using erasure codes can keep the same data recovery performance. Experimental results show that proposed method improves performance in storage efficiency 40%, read throughput 11%, write throughput 10% better than HDFS does.

A Case Study on Effective Technique of Distributed Data Storage for Big Data Processing in the Wireless Internet Environment

“…With the rapid increase of data scale, how to deal with big data has become a big issue. A large amount of storage space is necessary to store a massive amount of big data generated from the cloud storage [13]. To store and manage big data effectively, the cloud storage system uses distributed file system to connect several servers and nodes to network, which provides high availability and expandability [14].…”