SecDedoop: Secure Deduplication with Access Control of Big Data in the HDFS/Hadoop Environment

Ramya, P.; Sundar, C.

doi:10.1089/big.2019.0120

Cited by 11 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HDFS realizes the distributed storage of data, which solves a large number of problems existing in many distributed file systems. It includes the amount of data that can be stored (TB level or petabyte level), the reliability of storage, and better integration with Hadoop's Map Reduce framework [13].…”

Section: Hadoop Big Data Processingmentioning

confidence: 99%

Performance optimization of computing task scheduling based on the Hadoop big data platform

Hei

2022

Neural Comput & Applic

View full text Add to dashboard Cite

Hadoop, a distributed computing framework that can efficiently process large-scale datasets, has been used by an increasing number of organizations as the basic computing framework to build cloud computing platforms. Improving its execution efficiency is a hot research direction in the industry, and the scheduling problem is a key factor affecting the execution efficiency of Hadoop. It is very important to identify its shortcomings and improve them. This paper examines and analyses the optimization of computing task scheduling performance based on the Hadoop big data platform. This paper first analyses Hadoop big data processing. Hadoop has high scalability. Computing nodes can be added at any time, and they can participate in cluster work through simple configuration. The paper discusses the improvement in the Hadoop resource scheduling algorithm. The task scheduling algorithm in the Hadoop-based data task localization proposed in this paper is compared with the default algorithm used in the Hadoop task scheduling algorithm. The former shows better local data in all four jobs, there are more data localization tasks, and the expected goal is achieved. The effectiveness of the algorithm is verified, and the performance is improved by 30%.

show abstract

Section: Hadoop Big Data Processingmentioning

confidence: 99%

Performance optimization of computing task scheduling based on the Hadoop big data platform

Hei

2022

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Many researchers have also begun to use this framework to optimize and improve traditional machine learning algorithms. Some research has studied clustering algorithms under the Hadoop cloud platform [ 10 ]. Some research used the MapReduce distributed framework to parallelize the improvement of the traditional ant colony algorithm, which makes the traditional ant colony algorithm faster and more effective when processing large-scale data sets [ 11 , 12 ].…”

Section: Related Workmentioning

confidence: 99%

Visual Analysis of Sports Actions Based on Machine Learning and Distributed Expectation Maximization Algorithm

Luo¹

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In order to improve the scientificity of sports action analysis, this paper constructs a sports action analysis model based on machine learning based on the greedy algorithm and the bat algorithm. According to the structural characteristics of the model, the structure of the model is reflected in the form of face order, that is, the face neighborhood structure. Moreover, this paper judges the degree of similarity between model faces through the pros and cons of the order and applies it to the structural similarity matrix between models. In addition, this paper establishes corresponding mathematical models for the shape and structure of the model and constructs the shape similarity matrix, the surface neighborhood structure similarity matrix, and the structure similarity matrix between the source model and the target model. Finally, this paper designs and implements CAD model retrieval methods based on greedy algorithm and bat algorithm and designs experiments to compare the performance of the algorithm proposed in this paper with traditional algorithms. The result of the experiment shows that the algorithm proposed in this paper has obvious advantages in sports action analysis compared with the traditional algorithm.

show abstract

“…It is a measure of a test's accuracy and is known as the harmonic mean that conveys the balances between precision and recall. The F1-Score is calculated by using Equation (8).…”

Section: F1-measurementioning

confidence: 99%

“…MapReduce is a component of the Hadoop ecosystem that can process a large amount of information parallelly of the cluster of commodity hardware in a reliable manner. 8 Data processing is quite different in big data as compared with conventional data processing. The data are processed by using the divide-and-conquered approach to solve the assigned task.…”

Section: Introductionmentioning

confidence: 99%

An Ensemble-Based Scalable Approach for Intrusion Detection Using Big Data Framework

et al. 2021

View full text Add to dashboard Cite

In this study, we set up a scalable framework for large-scale data processing and analytics using the big data framework. The popular classification methods are implemented, tuned, and evaluated by using intrusion datasets. The objective is to select the best classifier after optimizing the hyper-parameters. We observed that the decision tree (DT) approach outperforms compared with other methods in terms of classification accuracy, fast training time, and improved average prediction rate. Therefore, it is selected as a base classifier in our proposed ensemble approach to study class imbalance. As the intrusion datasets are imbalanced, most of the classification techniques are biased toward the majority class. The misclassification rate is more in the case of the minority class. An ensemble-based method is proposed by using K-Means, RUSBoost, and DT approaches to mitigate the class imbalance problem; empirically investigate the impact of class imbalance on classification approaches' performance; and compare the result by using popular performance metrics such as Balanced Accuracy, Matthews Correlation Coefficient, and F-Measure, which are more suitable for the assessment of imbalanced datasets.

show abstract

SecDedoop: Secure Deduplication with Access Control of Big Data in the HDFS/Hadoop Environment

Cited by 11 publications

References 27 publications

Performance optimization of computing task scheduling based on the Hadoop big data platform

Performance optimization of computing task scheduling based on the Hadoop big data platform

Visual Analysis of Sports Actions Based on Machine Learning and Distributed Expectation Maximization Algorithm

An Ensemble-Based Scalable Approach for Intrusion Detection Using Big Data Framework

Contact Info

Product

Resources

About