Flow-Data Gathering Using NetFlow Sensors for Fitting Malicious-Traffic Detection Models

Abstract

Advanced persistent threats (APTs) are a growing concern in cybersecurity. Many companies and governments have reported incidents related to these threats. Throughout the life cycle of an APT, one of the most commonly used techniques for gaining access is network attacks. Tools based on machine learning are effective in detecting these attacks. However, researchers usually have problems with finding suitable datasets for fitting their models. The problem is even harder when flow data are required. In this paper, we describe a framework to gather flow datasets using a NetFlow sensor. We also present the Docker-based framework for gathering netflow data (DOROTHEA), a Docker-based solution implementing the above framework. This tool aims to easily generate taggable network traffic to build suitable datasets for fitting classification models. In order to demonstrate that datasets gathered with DOROTHEA can be used for fitting classification models for malicious-traffic detection, several models were built using the model evaluator (MoEv), a general-purpose tool for training machine-learning algorithms. After carrying out the experiments, four models obtained detection rates higher than 93%, thus demonstrating the validity of the datasets gathered with the tool.

Malicious traffic detection on sampled network flow data with novelty-detection-based models

“…Docker-based framework for gathering NetFlow data (DOROTHEA) is a tool that uses Docker as a base 13 . DOROTHEA 26 allows the creation of virtual networks with multiple machines and different structures to gather stream data.…”

“…MoEv has been used in many different research areas, such as in 43 , where the tool was used to detect jamming attacks in real-time location systems, and in 44 where the authors predicted academic success in educational institutions. Furthermore, in 13 , MoEV has been validated and used to detect network attacks. To validate the tool, the researchers replicated the work presented in 45 , obtaining similar results.…”

“…This paper also intends to demonstrate that detecting multiple network attacks in sampled network flows using anomaly detection-based techniques is possible. Finally, this work shows that the DOROTHEA tool 13 can generate NetFlow datasets with high sampling thresholds that can be used to train anomaly detection algorithms for their use on real traffic.…”

Enhanced Network Security against SQL Injection Attack Using Machine Learning

“…Two sets of NetFlow V5 data have been collected through the use of DOROTHEA, a Docker-based framework developed by Campazas-Vega et al (2020) [10]. This framework facilitates the creation of interconnected virtual networks for the purpose of generating and gathering flow data via the NetFlow protocol.…”