Flow-based Throughput Prediction using Deep Learning and Real-World Network Traffic

Hardegen, Christoph; Pfülb, Benedikt; Rieger, Sebastian; Gepperth, Alexander; Reismann, Sven

doi:10.23919/cnsm46954.2019.9012716

Cited by 19 publications

(13 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparing the results of a previously performed feature importance analysis [9] to the results presented in Section V-C, learning on a stream rather benefits from the enrichment but is context-dependent. While the initial feature importance experiments from [9] only consider a short time interval (10 blocks, 5 -10 minutes), the streaming experiments are based on a week's worth of flow data. Due to more network dynamics during the day, the accuracies are considerably less favorable than at night.…”

Section: B Results Of the Experimentsmentioning

confidence: 82%

“…In general, there is considerably more work-time network usage but a certain traffic level remains at night. In order to get an examplary overview of the traffic volume within the considered network as well as the trend of the flow cache sizes and the number of flow records exported per second for a complete week, we refer to [9]. c) Misbehavior of the NetFlow Exporter: While all generated and retraced flows [9] were correctly exported by the core switches in various tests, some flows were affected by a misbehavior of the NetFlow exporter during data collection.…”

Section: A Data Collectionmentioning

confidence: 99%

“…For example, we performed various downloads and compared the results with obtained flow information, e.g., a flow's duration and throughput. This is described in more detail in our previous work [9]. Figure 8 outlines the data distribution within the complete selected block for each flow label.…”

Section: Traffic Analysis Experimentsmentioning

confidence: 99%

“…The same applies to the differentiation between "short-term" and "long-term" flows [7]. As considered in [8] and [9], we specify a flow as metadata describing a stream of packets (≥ 1) that belongs to a coherent communication between a given source and destination system. 1) Problem Statement: In a classical routing scenario, the destination IP address only is used to select a link or path to forward a flow's packets.…”

Section: Introductionmentioning

confidence: 99%

“…Accordingly, anonymized flow data is used at all processing stages in the proposed pipeline to ensure privacy protection. This journal article is an extended version of the conference paper "Flow-based Throughput Prediction using Deep Learning and Real-World Network Traffic" from the Conference on Network and Service Management (CNSM) 2019 [9]. a) Main Contributions of the CNSM Paper:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Predicting Network Flow Characteristics Using Deep Learning and Real-World Network Traffic

Hardegen

Pfülb

Rieger

et al. 2020

IEEE Trans. Netw. Serv. Manage.

Self Cite

View full text Add to dashboard Cite

We present a processing pipeline for flow-based traffic classification using a machine learning component leveraging Deep Neural Networks (DNNs). The system is trained to predict likely characteristics of real-world traffic flows from a campus network ahead of time, e.g., a flow's throughput or duration. Training and evaluation of DNN models are continuously performed on a flow data stream collected from a university data center. Instead of the common binary classification into "mice" and "elephant" (throughput) or "short-term" and "long-term" (duration) flows, predicted flow characteristics are quantized into three classes. Various communication contexts (subset of network traffic, e.g., only TCP) and flow feature groups (subset of flow features, e.g., only a flow's 5-tuple), which are supported through an enrichment strategy, are considered and investigated. An in-depth description of the data acquisition process, including preprocessing steps and anonymization used to protect sensitive information, is given. Additionally, we employ an accelerated variant of t-distributed Stochastic Neighbor Embedding (t-SNE) to visualize network traffic data. This enables the understanding of traffic characteristics and relations between communication flows at a glance. Furthermore, possible use-cases and a highlevel architecture for flow-based routing scenarios utilizing the developed pipeline are proposed.

show abstract

Section: B Results Of the Experimentsmentioning

confidence: 82%

Section: A Data Collectionmentioning

confidence: 99%

Section: Traffic Analysis Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Predicting Network Flow Characteristics Using Deep Learning and Real-World Network Traffic

Hardegen

Pfülb

Rieger

et al. 2020

IEEE Trans. Netw. Serv. Manage.

Self Cite

View full text Add to dashboard Cite

show abstract

A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment

Kozik

Pawlicki

Choraś

2021

Pattern Anal Applic

View full text Add to dashboard Cite

The Internet of Things (IoT) appliances often expose sensitive data, either directly or indirectly. They may, for instance, tell whether you are at home right now or what your long or short-term habits are. Therefore, it is crucial to protect such devices against adversaries and has in place an early warning system which indicates compromised devices in a quick and efficient manner. In this paper, we propose time window embedding solutions that efficiently process a massive amount of data and have a low-memory-footprint at the same time. On top of the proposed embedding vectors, we use the core anomaly detection unit. It is a classifier that is based on the transformer’s encoder component followed by a feed-forward neural network. We have compared the proposed method with other classical machine-learning algorithms. Therefore, in the paper, we formally evaluate various machine-learning schemes and discuss their effectiveness in the IoT-related context. Our proposal is supported by detailed experiments that have been conducted on the recently published Aposemat IoT-23 dataset.

show abstract

A Frontier: Dependable, Reliable and Secure Machine Learning for Network/System Management

Zincir-Heywood

2020

J Netw Syst Manage

View full text Add to dashboard Cite

Flow-based Throughput Prediction using Deep Learning and Real-World Network Traffic

Cited by 19 publications

References 24 publications

Predicting Network Flow Characteristics Using Deep Learning and Real-World Network Traffic

Predicting Network Flow Characteristics Using Deep Learning and Real-World Network Traffic

A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment

A Frontier: Dependable, Reliable and Secure Machine Learning for Network/System Management

Contact Info

Product

Resources

About