Due to the explosive growth of data in the Internet, more and more applications are being deployed on Big Data platforms. However, as the scale of data continues to increase, the probability of anomalies in the platform is also increasing. However, traditional anomaly detection techniques cannot effectively handle the massive amount of historical data and can hardly meet the security requirements of big data platforms. In order to solve the above problems, this paper proposes a security anomaly detection method for big data platforms based on quantum optimization clustering. Firstly, a framework of big data platform anomaly detection system is designed based on distributed software architecture through Hadoop and Spark big data open source technology. The system achieves effective detection of network anomalies by collecting and analyzing big data platform server log data. Secondly, an offline anomaly detection algorithm based on quantum ant colony optimized affinity propagation clustering is designed for various anomalies mined from historical data. The bias parameters of the affinity propagation clustering are treated as individual ants to construct an ant colony, and the clustering accuracy is set as fitness. Finally, in order to improve the accuracy of the optimal path search of the ant colony, quantum bit encoding is applied to the ant colony position to refine the granularity of the individual ant colony position update. The experimental results show that the proposed method can effectively complete the anomaly clustering detection of massive data. With a reasonable threshold, the quantum ant colony–based affinity propagation clustering has high detection accuracy.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.