Combining Unsupervised Approaches for Near Real-Time Network Traffic Anomaly Detection

Abstract

The 0-day attack is a cyber-attack based on vulnerabilities that have not yet been published. The detection of anomalous traffic generated by such attacks is vital, as it can represent a critical problem, both in a technical and economic sense, for a smart enterprise as for any system largely dependent on technology. To predict this kind of attack, one solution can be to use unsupervised machine learning approaches, as they guarantee the detection of anomalies regardless of their prior knowledge. It is also essential to identify the anomalous and unknown behaviors that occur within a network in near real-time. Three different approaches have been proposed and benchmarked in exactly the same condition: Deep Autoencoding with GMM and Isolation Forest, Deep Autoencoder with Isolation Forest, and Memory Augmented Deep Autoencoder with Isolation Forest. These approaches are thus the result of combining different unsupervised algorithms. The results show that the addition of the Isolation Forest improves the accuracy values and increases the inference time, although this increase does not represent a relevant problematic factor. This paper also explains the features that the various models consider most important for classifying an event as an attack using the explainable artificial intelligence methodology called Shapley Additive Explanations (SHAP). Experiments were conducted on KDD99, NSL-KDD, and CIC-IDS2017 datasets.

A comprehensive investigation of clustering algorithms for User and Entity Behavior Analytics

“…Modeling a user profile, including their interests, characteristics, preferences, and behaviors, is a crucial step in defining a suitable baseline to feed into ML algorithms that are capable of predicting deviations from them that are not known a priori (Eke et al, 2019 ; Savenkov and Ivutin, 2020 ). Unsupervised learning algorithms are useful when there is no prior knowledge of the anomaly being investigated (Vikram and Mohana, 2020 ; Carrera et al, 2022 ; Mochurad et al, 2023 ). A UEBA engine can greatly benefit from unsupervised learning algorithms because any substantial deviation from normal behavior in common communication patterns can represent a potential attack (Martín et al, 2021 ; Fysarakis et al, 2023 ).…”

Extraction of Time-Domain Characteristics and Selection of Effective Features Using Correlation Analysis to Increase the Accuracy of Petroleum Fluid Monitoring Systems

“…The used validation methods and training processes in this paper are well-known methods in most modeling and optimization problems, which have been used in many studies [110][111][112][113][114][115]. In several studies [116][117][118][119][120][121][122][123][124][125], different mathematical methods such as feature extraction, feature reduction, feature selection, correlation analysis, and numerical calculation, etc., have been used. In this study, feature extraction in the time domain and correlation analysis were used in order to present a novel metering system.…”