Dynamic Deduplication Algorithm for Cross-User Duplicate Data in Hybrid Cloud Storage

Abstract

The escalating growth of distributed big data in hybrid cloud storage architecture introduces a new set of challenges. Constantly, content enrichment puts pressure on capacity. Nonetheless, the explosion of user data places a significant strain on broadband and storage capacity. Consequently, many cloud storage providers will implement deduplication to compress data, reduce transfer bandwidth, and reduce cloud storage space. In cloud storage systems, it is a data compression and storage optimization method. By locating and removing redundant data, it can save storage space and bandwidth. An MTHDedup deduplication strategy based on the Merkle hash tree is presented in a hybrid cloud environment to address the issue of convergent encryption algorithms being susceptible to brute-force attacks and ciphertext computation time overhead. Merkle hash trees are constructed using additional encryption algorithms to generate encryption keys during file- and block-level deduplication, ensuring that generated ciphertexts are unpredictable. The method is effective against both internal and external brute-force attacks, thereby increasing data security. Our method reduces the computational burden of ciphertext generation and the key storage space, and the performance advantage increases with the number of privilege sets.

Revolutionary Data Deduplication With Fuzzy C-means: Advancing Data Quality Management

“…The economic model for Hybrid Encrypted Cloud Data Deduplication (H-DEDU) (Ref [11]) introduces an intriguing perspective by incorporating economic considerations into the decision-making process for stakeholders. MTHDedup's deduplication strategy (Ref [12]) stands out for its resilience against internal and external attacks through the use of Merkle hash trees. The hybrid deduplication technique using Euclidean distance and chromatic correlation clustering (Ref [13]) addresses the limitations of existing methods, presenting a cost-effective and time-e cient approach.…”

“…In ref [12] Presents an MTHDedup deduplication method based totally on Merkle hash bushes to deal with convergent encryption vulnerabilities. The approach complements records security by constructing Merkle hash timber and the usage of additional encryption algorithms at some stage in the report-and block-degree deduplication.…”

“…The [11]establishes an economic model and multi-degree sport for Hybrid Encrypted Cloud Data Deduplication, analyzing Nash Equilibrium conditions and featuring a gradient-primarily based set of rules for the most bene cial approach selection. [12]introduces MTHDedup, a strategy based on Merkle hash trees for enhanced facts safety in hybrid cloud garages. Lastly, [13] proposes a hybrid deduplication approach using crowdsourcing and Euclidean distance, aiming to lessen costs, decrease deduplication time, and enhance accuracy, outperforming current techniques.…”

RESIST: Randomized Encryption for Deduplicated Cloud Storage System

“…An additional key managing server is used to generate encryption keys. For block-level deduplication, Gang et al [20] proposed a Merkle tree based deduplication model. This scheme deploys an additional server to compute blocklevel encryption keys.…”

“…In this way, convergent encryption allows the deduplication of the encrypted files. Several recent state-of-the-art schemes [18,20,24,25,46,47,52] present variants of convergent encryption. However, these schemes are vulnerable to dictionary attacks as they use deterministic encryption.…”