Hadoop-based secure storage solution for big data in cloud computing environment

Guan, Shaopeng; Zhang, Conghui; Wang, Yilin; Liu, Wenqing

doi:10.1016/j.dcan.2023.01.014

Cited by 13 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it failed to decrypt the user data and validate the owner uploads by employing the signature and verification phase. Guan, S.,et al [7] developed a Hadoop-based big data secure storage scheme for big data in cloud computing. This model was highly reliable in providing solutions to single-point metadata failures and solved the bottleneck issues caused by single-name node servers, but homomorphic encryption algorithms were not utilized by this approach to increase data security in the cloud computing environment.…”

Section: Literature Reviewmentioning

confidence: 99%

IABHE: Identity and Attribute-based Honey Encryption Algorithm and Deep Learning-based Key Generation for Securing Big Data in Hadoop Framework

Siyal,

Long

2024

Preprint

View full text Add to dashboard Cite

Nowadays, Hadoop is considered promising to store and reliably process big data in the cloud computing platform, and it also provides low-cost and flexible services for large amounts of data. However, the possibility of malicious attacks on stored or processed data has increased due to the absence of inherent security mechanisms in Hadoop. In this approach, an Identity and Attribute-Based Honey Encryption Algorithm (IABHE) is developed for the encryption of data to offer security to the big data in the Hadoop framework. Here, the system model of the cloud is considered initially. Then, acquired input big data is applied to the MapReduce framework, where data encryption is performed by IABHE, which incorporates identity and attribute-based encryption with honey encryption. Moreover, a deep learning model, PyramidNet is utilized for the generation of secret keys used in encryption. Moreover, the encrypted data obtained in the mapper phase is aggregated by performing polynomial interpolation in the reducer phase. Finally, encrypted data is securely saved in the cloud. Here, the efficiency of IABHE is evaluated utilizing various performance metrics and it attained better values of encryption time at 17.756sec, decryption time at 8.767sec, and key complexity at 0.368.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

IABHE: Identity and Attribute-based Honey Encryption Algorithm and Deep Learning-based Key Generation for Securing Big Data in Hadoop Framework

Siyal,

Long

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…In Hadoop, because the NameNode manages the metadata information of the entire HDFS file system, once a single NameNode stops working in the cluster, it will affect the real-time operation of the entire cluster [15][16], making the cluster unable to process data in real time, and it is also not conducive to the cluster upgrade and maintenance, the NameNode is designed in a high-availability mode. Using the high-availability cluster design, even if a master node fails, the standby node can be switched to the working mode immediately through ZKFC (ZKFailoverController) [36][37] process to perform failover and ensure the normal operation of the cluster. Similarly, the ResourceManager, which manages cluster resources, is also designed for high availability in this paper.…”

Section: Spark Distributed Cluster Based On Highly Available Hadoopmentioning

confidence: 99%

Simulation Research on Fast Matching of Big Data Based on Spark

Song

Leng

et al. 2023

IEEE Access

View full text Add to dashboard Cite

To solve the problem of low efficiency in real-time processing and matching of CNAME records in massive DNS log data, a parallel AC automaton enhancement method based on Spark was proposed. The method is based on the Spark distributed cluster computing engine of Hadoop, which ensures the stability of massive DNS log data storage with high fault tolerance and 24-hour real-time processing. At the same time, the Spark distributed cluster uses the multi-thread parallel computing method combined with the improved AC automaton algorithm, which not only reduces the memory occupied by trie construction, but also improves the efficiency of rapid matching of CNAME records of massive DNS logs. Simulation results show that the proposed method can quickly match CNAME records of massive DNS log data. Compared with the original AC algorithm, the efficiency is significantly improved, and the time complexity and storage space are reduced.

show abstract

“…The paper explored the application of this encryption method to protect the confidentiality and integrity of big data. S. Guan et al (2023) presented a secure storage solution for big data in a cloud computing environment using Hadoop. The paper focused on ensuring the security and privacy of big data in cloud-based storage systems.…”

Section: Poet In Hyperledger Sawtoothmentioning

confidence: 99%

Blockchain Enabled Hadoop Distributed File System Framework for Secure and Reliable Traceability

Gupta,

Dwivedi

2023

ADCAIJ

View full text Add to dashboard Cite

Hadoop Distributed File System (HDFS) is a distributed file system that allows large amounts of data to be stored and processed across multiple servers in a Hadoop cluster. HDFS also provides high throughput for data access. HDFS enables the management of vast amounts of data using commodity hardware. However, security vulnerabilities in HDFS can be manipulated for malicious purposes. This emphasizes the significance of establishing strong security measures to facilitate file sharing within Hadoop and implementing a reliable mechanism for verifying the legitimacy of shared files. The objective of this paper is to enhance the security of HDFS by utilizing a blockchain-based technique. The proposed model uses the Hyperledger Fabric platform at the enterprise level to leverage metadata of files, thereby establishing dependable security and traceability of data within HDFS. The analysis of results indicates that the proposed model incurs a slightly higher overhead compared to HDFS and requires more storage space. However, this is considered an acceptable trade-off for the improved security.

show abstract

Hadoop-based secure storage solution for big data in cloud computing environment

Cited by 13 publications

References 17 publications

IABHE: Identity and Attribute-based Honey Encryption Algorithm and Deep Learning-based Key Generation for Securing Big Data in Hadoop Framework

IABHE: Identity and Attribute-based Honey Encryption Algorithm and Deep Learning-based Key Generation for Securing Big Data in Hadoop Framework

Simulation Research on Fast Matching of Big Data Based on Spark

Blockchain Enabled Hadoop Distributed File System Framework for Secure and Reliable Traceability

Contact Info

Product

Resources

About