The detection of anomalies in data is a far-reaching field of research which also applies to the field of cloud computing in several different ways: from the detection of various types of intrusions to the detection of hardware failures, many publications address how far anomaly detection methods are able to meet the specific requirements of a cloud-based network. Since there is still no comprehensive overview of this constantly growing field of research, this literature review provides a systematic evaluation of 215 publications that can be considered as representative for the last ten years of this scientific development. Our analysis identifies three main methodological areas (machine learning, deep learning, statistical approaches) and summarizes how exactly the corresponding models are applied for the detection of anomalies. Furthermore, we clarify which concrete application areas are typically addressed by anomaly detection in the context of cloud computing environments and which related public datasets are often used for evaluations. Finally, we discuss the implications of the literature review and provide directions for future research.
The detection of anomalies in cloud metrics is an important way to identify suspicious data instances that indicate a system problem such as hardware failures, performance bottlenecks or intrusions. Yet, especially in a cloud computing infrastructure where the amount and variety of services is constantly increasing, it is getting more and more challenging to monitor and maintain the system manually. Thus, it is beneficial to use machine learning to detect anomalies at least partially in an automated way. The contribution of this paper is two-folded: firstly, we evaluate three unsupervised, reconstruction-based methods for anomaly detection (PCA, Autoencoder, LSTM-Encoder-Decoder) on the Yahoo! Webscope S5 benchmark dataset. Secondly, we compare our chosen models to a widely-used density-based approach and show that our reconstruction-based approaches outperform the related work. CCS CONCEPTS • Computing methodologies → Anomaly detection; • Networks → Cloud Computing; Network monitoring.
The need to detect bias in machine learning (ML) models has led to the development of multiple bias detection methods, yet utilizing them is challenging since each method: i) explores a different ethical aspect of bias, which may result in contradictory output among the different methods, ii) provides an output of a different range/scale and therefore, can't be compared with other methods, and iii) requires different input, and therefore a human expert needs to be involved to adjust each method according to the examined model. In this paper, we present BENN-a novel bias estimation method that uses a pretrained unsupervised deep neural network. Given a ML model and data samples, BENN provides a bias estimation for every feature based on the model's predictions. We evaluated BENN using three benchmark datasets and one proprietary churn prediction model used by a European Telco and compared it with an ensemble of 21 existing bias estimation methods. Evaluation results highlight the significant advantages of BENN over the ensemble, as it is generic (i.e., can be applied to any ML model) and there is no need for a domain expert, yet it provides bias estimations that are aligned with those of the ensemble.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.