Cost-Sensitive Distributed Machine Learning for NetFlow-Based Botnet Activity Detection

Kozik, Rafał; Pawlicki, Marek; Choraś, Michał

doi:10.1155/2018/8753870

Cited by 18 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of these algorithms was dictated by the following factors: Random Forest has been proven in multiple studies on network attacks; its performance was always high [ 4 ], and results were satisfactory, and the authors have found promising results from the utilization of this algorithm in earlier work [ 54 , 55 ]. The Gradient Boosted Trees (GBT) algorithm combines the advantages of RandomForest with the added benefit of gradient utilization.…”

Section: Experiments and Resultsmentioning

confidence: 99%

The Proposition and Evaluation of the RoEduNet-SIMARGL2021 Network Intrusion Detection Dataset

Mihailescu

Mihai

Carabaș

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Cybersecurity is an arms race, with both the security and the adversaries attempting to outsmart one another, coming up with new attacks, new ways to defend against those attacks, and again with new ways to circumvent those defences. This situation creates a constant need for novel, realistic cybersecurity datasets. This paper introduces the effects of using machine-learning-based intrusion detection methods in network traffic coming from a real-life architecture. The main contribution of this work is a dataset coming from a real-world, academic network. Real-life traffic was collected and, after performing a series of attacks, a dataset was assembled. The dataset contains 44 network features and an unbalanced distribution of classes. In this work, the capability of the dataset for formulating machine-learning-based models was experimentally evaluated. To investigate the stability of the obtained models, cross-validation was performed, and an array of detection metrics were reported. The gathered dataset is part of an effort to bring security against novel cyberthreats and was completed in the SIMARGL project.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

The Proposition and Evaluation of the RoEduNet-SIMARGL2021 Network Intrusion Detection Dataset

Mihailescu

Mihai

Carabaș

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors of [22] used three distributed algorithms-extreme learning machines (ELM), distributed random forest, and distributed random boosted-trees-to detect botnet attacks. In the research paper, they presented the concepts and architecture of the system, which was based on big query data processing.…”

Section: Related Workmentioning

confidence: 99%

How to Effectively Collect and Process Network Data for Intrusion Detection?

Komisarek

Pawlicki

Kozik

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

The number of security breaches in the cyberspace is on the rise. This threat is met with intensive work in the intrusion detection research community. To keep the defensive mechanisms up to date and relevant, realistic network traffic datasets are needed. The use of flow-based data for machine-learning-based network intrusion detection is a promising direction for intrusion detection systems. However, many contemporary benchmark datasets do not contain features that are usable in the wild. The main contribution of this work is to cover the research gap related to identifying and investigating valuable features in the NetFlow schema that allow for effective, machine-learning-based network intrusion detection in the real world. To achieve this goal, several feature selection techniques have been applied on five flow-based network intrusion detection datasets, establishing an informative flow-based feature set. The authors’ experience with the deployment of this kind of system shows that to close the research-to-market gap, and to perform actual real-world application of machine-learning-based intrusion detection, a set of labeled data from the end-user has to be collected. This research aims at establishing the appropriate, minimal amount of data that is sufficient to effectively train machine learning algorithms in intrusion detection. The results show that a set of 10 features and a small amount of data is enough for the final model to perform very well.

show abstract

“…Through a recursive query, malicious attackers resolve normal DNS resolution requests to their malicious servers [5], [6]. In this process, malicious attackers apply domain-flux or fast-flux technique to locate their Command and Control (C&C) server by automatically generating a large number of non-existent domain names using domain generation algorithms (DGA) [7]- [11].…”

Section: Introductionmentioning

confidence: 99%

Malicious Domain Names Detection Algorithm Based on Lexical Analysis and Feature Quantification

et al. 2019

View full text Add to dashboard Cite

Malicious domain names usually refer to a series of illegal activities, posing threats to people's privacy and property. Therefore, the problem of detecting malicious domain names has aroused widespread concerns. In this study, a malicious domain names detection algorithm based on lexical analysis and feature quantification is proposed. To achieve efficient and accurate detection, the method includes two phases. The first phase checks an observed domain name against a blacklist of known malicious uniform resource locator (URLs). The observed domain name is classified as being definitely malicious or potentially malicious based on its edit distances to the domain names on the blacklist. The second phase further evaluates a potential malicious domain name by its reputation value that represents its lexical feature and is calculated based on an N-gram model. The top 100,000 normal domain names in Alexa are used to obtain a whitelist substring set using the N-gram method in which each domain name excluding the top-level domain is segmented into substrings with the length of 3, 4, 5, 6 and 7. The weighted values of the substrings are calculated according to their occurrence counts in the whitelist substring set. A potential malicious domain name is segmented by the N-gram method and its reputation value is calculated based on the weighted values of its substrings. Finally, the potential malicious domain name is determined to be malicious or normal based on its reputation value. The effectiveness of the proposed detection method has been demonstrated by experiments on public available data.INDEX TERMS Malicious domain names, N-gram, domain name substring, edit distance, reputation value.

show abstract

Cost-Sensitive Distributed Machine Learning for NetFlow-Based Botnet Activity Detection

Cited by 18 publications

References 12 publications

The Proposition and Evaluation of the RoEduNet-SIMARGL2021 Network Intrusion Detection Dataset

The Proposition and Evaluation of the RoEduNet-SIMARGL2021 Network Intrusion Detection Dataset

How to Effectively Collect and Process Network Data for Intrusion Detection?

Malicious Domain Names Detection Algorithm Based on Lexical Analysis and Feature Quantification

Contact Info

Product

Resources

About