Detecting Anomalies in Encrypted Traffic via Deep Dictionary Learning

Xing, Junchi; Wu, Chunming

doi:10.1109/infocomwkshps50562.2020.9162940

Cited by 16 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Packet statistics [28] Payload statistics [6] Flow level statistics [36] Automatic Feature Extraction Packet header [25], [34] ✓ Packet interval [44], [32] OADSD feature extraction adaptable to the environment.…”

Section: Domain Expert Featuresmentioning

confidence: 99%

“…Malicious traffic detection. Online Traffic Anomaly Detection methods have been widely investigated [28], [30], [31], [2], [44]. Online Traffic Anomaly Detection methods are widely investigated [28], [30], [31], [2], [44].…”

Section: Testbed Evaluationmentioning

confidence: 99%

“…Online Traffic Anomaly Detection methods have been widely investigated [28], [30], [31], [2], [44]. Online Traffic Anomaly Detection methods are widely investigated [28], [30], [31], [2], [44]. [31], [30] combine the online and offline machine learning methods to make the methods easier to train and can identify new traffic based on the prediction result of online and offline algorithms.…”

Section: Testbed Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Zhang

et al. 2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Software Defined Network (SDN) has been widely used in modern network architecture. The SD-WAN is considered as a technology that has a potential to revolutionize the WAN service usage by utilizing the SDN philosophy. Attacking SDN router and controller can affect the network and block the entire services. In this paper, we propose a machine learning based anomalous traffic detection framework named OADSD over SD-WAN that can achieve task independent and has the ability of adapting to the environment. The OADSD adopts Distributed Dynamic Feature Extraction (DDFE) to extract representative features directly from the raw traffic, and proposes the Ondemand Evolving Isolation Forest (OEIF) to make the system adapt to an environment. We provide a theoretical analysis of the performance of the OADSD. We also conduct comprehensive experiments to evaluate the performance of the OADSD with real world public datasets as well as a small real testbed. Our experiments under real world public datasets show that, the OADSD can accurately detect various kinds of attacks with a high performance. Compared with the state-of-the-art systems, the OADSD can achieve up to 60% accuracy improvement.

show abstract

Section: Domain Expert Featuresmentioning

confidence: 99%

Section: Testbed Evaluationmentioning

confidence: 99%

Section: Testbed Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Zhang

et al. 2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…The authors conducted comparative experiments with KNN and other existing works which used the VPN non-VPN [55] and CICIDS 2012 [54] datasets to demonstrate that the proposed DFR is more accurate and robust. Xing et al [10] proposed an online detection model based on deep dictionary learning, D2LAD, to address the noisy data label, long training time and high traffic data distribution variance. The model can learn and extract sequential features from raw traffic data input based on a pre-trained LSTM auto-encoder.…”

Section: E Algorithms Selectionmentioning

confidence: 99%

Machine Learning for Encrypted Malicious Traffic Detection: Approaches, Datasets and Comparative Study

Wang,

Fok,

Thing

2022

Preprint

View full text Add to dashboard Cite

As people's demand for personal privacy and data security becomes a priority, encrypted traffic has become mainstream in the cyber world. However, traffic encryption is also shielding malicious and illegal traffic introduced by adversaries, from being detected. This is especially so in the post-COVID-19 environment where malicious traffic encryption is growing rapidly. Common security solutions that rely on plain payload content analysis such as deep packet inspection are rendered useless. Thus, machine learning based approaches have become an important direction for encrypted malicious traffic detection. In this paper, we formulate a universal framework of machine learning based encrypted malicious traffic detection techniques and provided a systematic review. Furthermore, current research adopts different datasets to train their models due to the lack of well-recognized datasets and feature sets. As a result, their model performance cannot be compared and analyzed reliably. Therefore, in this paper, we analyse, process and combine datasets from 5 different sources to generate a comprehensive and fair dataset to aid future research in this field. On this basis, we also implement and compare 10 encrypted malicious traffic detection algorithms. We then discuss challenges and propose future directions of research.

show abstract

“…For example, crypto-mining attacks can be traced by monitoring the abnormal CPU usage (in normal PC) in the resource usage logs or a massive number of communications with other peer networks. Moreover, given the popularity of secure networks and webs (e.g., using HTTPS), it is a challenge to analyze the encrypted traffic payloads [17].…”

mentioning

confidence: 99%

Machine Learning With Variational AutoEncoder for Imbalanced Datasets in Intrusion Detection

et al. 2022

View full text Add to dashboard Cite

As a result of the explosion of security attacks and the complexity of modern networks, machine learning (ML) has recently become the favored approach for intrusion detection systems (IDS). However, the ML approach usually faces three challenges: massive attack variants, imbalanced data issues, and appropriate data segmentation. Improper handling of the issues will significantly degrade ML performance, e.g., resulting in high false-negative and low recall rates. Despite many efforts have done in the literature, detecting security attacks in a complicated network environment with imperfect data collection is still an open issue. This work proposes a machine learning framework with a combination of variational autoencoder and multilayer perceptron to deal with imbalanced datasets and detecting the explosion of attack variants on the Internet. The detection engine also includes an efficient range-based sequential search algorithm to address the segmentation challenge in data pre-processing from multiple sources (network packets, system/statistic logs) effectively. Our work is the first attempt to demonstrate the effect of using an appropriate combination of ML models for boosting IDS detection capability in a heterogeneous environment, where data collection imperfection is common. Experimental results on a public system log dataset (e.g., HDFS) show that our method gains approximately as much as 97% on F1 score and 98% on recall rate, a promising result compared to the same measurement of other solutions. Even better, we found that the proposed treatment of imbalanced datasets can improve up to 35% on the F1 score and 27% on recall rate. The testing results also indicate that our model can detect new attack variants. Code is available at https://github.com/tuonglinhhm/Hybrid-Learning-AutoEncoder-IDS.

show abstract

Detecting Anomalies in Encrypted Traffic via Deep Dictionary Learning

Cited by 16 publications

References 18 publications

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Machine Learning for Encrypted Malicious Traffic Detection: Approaches, Datasets and Comparative Study

Machine Learning With Variational AutoEncoder for Imbalanced Datasets in Intrusion Detection

Contact Info

Product

Resources

About