Towards Evaluating Quality of Datasets for Network Traffic Domain

Soukup, Dominik; Tisovcik, Peter; Hynek, Karel; Čejka, Tomáš

doi:10.23919/cnsm52442.2021.9615601

Cited by 8 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Initial definitions related to the quality of datasets are provided by Soukup et al [38]. The paper describes some key terms, including the difference between the quality of data and the quality of datasets.…”

Section: Related Workmentioning

confidence: 99%

Dataset Quality Assessment with Permutation Testing Showcased on Network Traffic Datasets

Wasielewska¹,

Soukup²,

Čejka³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Intelligent and autonomous networks require precise and fast mechanisms that ensure error-free and efficient operation. Modern solutions are increasingly based on artificial intelligence, in particular on machine learning, to reliably process huge amounts of data. Therefore, high-quality datasets are essential to train machine learning models. Unfortunately, the problem of assessing the quality of datasets is very challenging and often overlooked. This paper proposes a method for assessing the dataset quality in the context of binary classification. It is based on permutation testing and examines the strength of the relationship between observations and labels. Experiments carried out on simulated and real network datasets show that the method is sensitive to detect errors/mislabels in the labeled dataset. We also present theoretical considerations justifying our results.</p>

show abstract

Section: Related Workmentioning

confidence: 99%

Dataset Quality Assessment with Permutation Testing Showcased on Network Traffic Datasets

Wasielewska¹,

Soukup²,

Čejka³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Quality of datasets is important area for real deployment of ML classifiers. After initial definitions from Soukup et al [5], Camacho et al [11] proposed a permutation based network dataset quality assessment with promising results. Li et al [12] proposed a distance-entropy method to distinguish the good and bad data.…”

Section: B Quality Of Datasetsmentioning

confidence: 99%

“…Nevertheless, this would incrementally increase the size of the dataset, its redundancy and complexity. Therefore in the proposed ALF, we include an option for dataset optimization methods that remove unnecessary records and try to find the minimal dataset as described by Soukup et al [5].…”

Section: Fig 1: Example Of Active Learning Framework Workflowmentioning

confidence: 99%

“…These classes are cross-referenced and exchange data via class attributes. Based on this approach, ALF can be used as a standalone module but can also be integrated with other frameworks, e.g., NEMEA 5 [1] is a working example.…”

Section: Fig 1: Example Of Active Learning Framework Workflowmentioning

confidence: 99%

“…• We showcase experiments on real data and use cases to prove feasibility of ALF. This paper is a broad extension of our previous work [5], which provided initials ideas for quality of datasets and ML model deployment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active Learning Framework to Automate Network Traffic Classification

Pesek¹,

Soukup²,

Čejka³

2022

Preprint

View full text Add to dashboard Cite

<p>Recent network traffic classification methods benefit from machine learning (ML) technology. However, there are many challenges due to use of ML, such as: lack of high-quality annotated datasets, data-drifts and other effects causing aging of datasets and ML models, high volumes of network traffic etc. This paper argues that it is necessary to augment traditional workflows of ML training&deployment and adapt Active Learning concept on network traffic analysis. The paper presents a novel Active Learning Framework (ALF) to address this topic. ALF provides prepared software components that can be used to deploy an active learning loop and maintain an ALF instance that continuously evolves a dataset and ML model automatically. The resulting solution is deployable for IP flow-based analysis of high-speed (100 Gb/s) networks, and also supports research experiments on different strategies and methods for annotation, evaluation, dataset optimization, etc. Finally, the paper lists some research challenges that emerge from the first experiments with ALF in practice.</p>

show abstract