Cost-Effective Data Placement in Edge Storage Systems With Erasure Code

Abstract

Edge computing, as a new computing paradigm, brings cloud computing's computing and storage capacities to network edge for providing low latency services for users. The networked edge servers in a specific area constitute edge storage systems (ESSs), where popular data can be stored to serve the users in the area. The novel ESSs raise many new opportunities as well as unprecedented challenges. Most existing studies of ESSs focus on the storage of data replicas in the system to ensure low data retrieval latency for users. However, replica-based edge storage strategies can easily incur high storage costs. It is not cost-effective to store massive replicas of large-size data, especially those that do not require real-time access at the edge, e.g., system upgrade files, popular app installation files, videos in online games. It may not even be possible due to the constrained storage resources on edge servers. In this paper, we make the first attempt to investigate the use of erasure codes in cost-effective data storage at the edge. The focus is to find the optimal strategy for placing coded data blocks on the edge servers in an ESS, aiming to minimize the storage cost while serving all the users in the system. We first model this novel Erasure Coding based Edge Data Placement (EC-EDP) problem as an integer linear programming problem and prove its N P-hardness. Then, we propose an optimal approach named EC-EDP-O based on integer programming. Another approximation algorithm named EC-EDP-V is proposed to address the high computation complexity of large-scale EC-EDP scenarios efficiently. The extensive experimental results demonstrate that EC-EDP-O and EC-EDP-V can save an average of 68.58% (and up to 81.16% in large-scale scenarios) storage cost compared with replica-based storage approaches.

Cognitive Update and Fast Recovery Using Machine Learning and Parity Delta Shareability for Erasure-Coded Storage Systems

“…File data is segmented into multiple data chunks before being encoded to generate parity chunks in erasure-coded storage systems [25,26]. The data and parity chunks are then uniformly disseminated across various storage servers.…”

Enabling Balanced Data Deduplication in Mobile Edge Computing

“…This is equivalent to Constraint (3), i.e., all the edge servers covered by data d i should still be covered after deduplication. Constraint (16) specifies the domains of η j and τ j i . It is the same as the constraint for binary variable r j i .…”

“…Therefore, EDD aims to remove duplicate data at the file level rather than the chunk level. In addition, data retrieval between edge servers must not violate the latency constraint -an edge server can only retrieve data from edge servers within its latency limitation, i.e., their nearby edge servers in the ESS [1], [16], [17]. Thus, EDD must ensure that after data deduplication, all the users can still retrieve requested data under the latency constraint.…”

RESS: A Reliable and Effcient Storage Scheme for Bitcoin Blockchain Based on Raptor Code

“…That is, to determine whether the transaction used for "payment" has been verified by the full node, and to be protected by how much computing power (how many confirmations). Due to the lack of complete transaction records, light nodes cannot verify that a transaction does not exist, and this vulnerability can easily lead to denial of service attacks and double-spending attacks [18]. In addition, light nodes need to randomly link multiple nodes, increasing the probability of connecting with at least one reliable node, but this random link requirement is also vulnerable to network partitioning and sybil attacks.…”