DGA Botnet detection using Collaborative Filtering and Density-based Clustering

Nguyen, Trung-Duc; Cao, Tuan‐Dung; Nguyen, Linh-Giang

doi:10.1145/2833258.2833310

Cited by 18 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [14], the method based on network flow information over DNS traffic rather than domain names was proposed, but it is limited by the difficulty of collecting the flow information in large-scale networks. In [15], offline analysis to detect DGA botnets through whitelist filtering and clustering was given. In [16], BotMeter, a tool that charts the DGA-bot population landscapes in large-scale networks was proposed, which relies on a long period of analysis.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Word-Based Algorithmically Generated Domains Using Semantic Analysis

et al. 2019

View full text Add to dashboard Cite

In highly sophisticated network attacks, command-and-control (C&C) servers always use domain generation algorithms (DGAs) to dynamically produce several candidate domains instead of static hard-coded lists of IP addresses or domain names. Distinguishing the domains generated by DGAs from the legitimate ones is critical for finding out the existence of malware or further locating the hidden attackers. The word-based DGAs disclosed in recent network attack events have shown significantly stronger stealthiness when compared with traditional character-based DGAs. In word-based DGAs, two or more words are randomly chosen from one or more specific dictionaries to form a dynamic domain, these regularly generated domains aim to mimic the characteristics of a legitimate domain. Existing DGA detection schemes, including the state-of-the-art one based on deep learning, still cannot find out these domains accurately while maintaining an acceptable false alarm rate. In this study, we exploit the inter-word and inter-domain correlations using semantic analysis approaches, word embedding and the part-of-speech are taken into consideration. Next, we propose a detection framework for word-based DGAs by incorporating the frequency distribution of the words and that of part-of-speech into the design of the feature set. Using an ensemble classifier constructed from Naive Bayes, Extra-Trees, and Logistic Regression, we benchmark the proposed scheme with malicious and legitimate domain samples extracted from public datasets. The experimental results show that the proposed scheme can achieve significantly higher detection accuracy for word-based DGAs when compared with three state-of-the-art DGA detection schemes.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In actual fact, the methods in [12][13][14][15][16] are all limited by the status of the network environment and data integrity. In real networks, especially in large-scale networks, these traffic features are very difficult to collect.…”

Section: Related Workmentioning

confidence: 99%

Detecting Word-Based Algorithmically Generated Domains Using Semantic Analysis

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Reference Building accurate and practical recommender system algorithms using machine learning classifier and collaborative filtering [20] DGA botnet detection using collaborative filtering and density-based clustering [21] A multistage collaborative filtering method for fall detection [22] Analysis and performance of collaborative filtering and classification algorithms [1] Extracting a vocabulary of surprise by collaborative filtering mixture and analysis of feelings [4] Content based filtering in online social network using inference algorithm [23] Building switching hybrid recommender system using machine learning classifiers and collaborative filtering [8] Imputation-boosted collaborative filtering using machine learning classifiers [24] CRISP-an interruption management algorithm based on collaborative filtering [25] A credit scoring model based on collaborative filtering [26] Collaborative filtering recommender systems [2] An improved switching hybrid recommender system using naive Bayes classifier and collaborative filtering [6] Tweet modeling with LSTM recurrent neural networks for hashtag recommendation [27] A two-stage cross-domain recommendation for cold start problem in cyber-physical systems [28] ELM based imputation-boosted proactive recommender systems [29] Twitter-user recommender system using tweets: a content-based approach [30] A personalized time-bound activity recommendation system [31] Automated content based short text classification for filtering undesired posts on Facebook [32] Shilling attack detection in collaborative recommender systems using a meta learning strategy [33] Building a distributed generic recommender using scalable data mining library [34] Context-aware movie recommendation based on signal processing and machine learning [35] Recommender systems using linear classifiers [36] A survey of accuracy evaluation metrics of recommendation tasks [3] Incorporating user control into recommender systems based on naive Bayesian classification [37] Classification features for attack detection in collaborative recommender systems [38] Automatic tag recommendation algorithms for social recommender systems [39] Optimizing similar item recommendations in a semistructured marketplace to maximize conversion …”

Section: Titlementioning

confidence: 99%

Recommendation and Classification Systems: A Systematic Mapping Study

Enríquez

Morales-Trujillo

Calle-Alonso

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

Today, recommendation algorithms are widely used by companies in multiple sectors with the aim of increasing their profits or offering a more specialized service to their customers. Moreover, there are countless applications in which classification algorithms are used, seeking to find patterns that are difficult for people to detect or whose detection cost is very high. Sometimes, it is necessary to use a mixture of both algorithms to give an optimal solution to a problem. This is the case of the ADAGIO, a R&D project that combines machine learning (ML) strategies from heterogeneous data sources to generate valuable knowledge based on the available open data. In order to support the ADAGIO project requirements, the main objective of this paper is to provide a clear vision of the existing classification and recommendation ML systems to help researchers and practitioners to choose the best option. To achieve this goal, this work presents a systematic review applied in two contexts: scientific and industrial. More than a thousand papers have been analyzed resulting in 80 primary studies. Conclusions show that the combination of these two algorithms (classification and recommendation) is not very used in practice. In fact, the validation presented for both cases is very scarce in the industrial environment. From the point of view of software development life cycle, this review also shows that the work being done in the ML (for classification and recommendation) research and industrial environment is far from earlier stages such as business requirements and analysis. This makes it very difficult to find efficient and effective solutions that support real business needs from an early stage. It is therefore that the article suggests the development of new ML research lines to facilitate its application in the different domains.

show abstract

“…Trong các công trình [1,10,11], nhóm tác giả cũng đề xuất phương pháp phát hiện DGA botnet sử dụng đặc trưng ngữ nghĩa, đặc trưng thống kê, áp dụng các phương pháp phân cụm dựa trên mật độ DBSCAN, lọc cộng tác (collaborative filtering) và map-reduce để tính độ tương hợp giữa các hành vi của các máy trạm, K-means và biến thể của khoảng cách Mahalanobis. Tuy nhiên những cách tiếp cận này này thường chỉ hiệu quả với một hoặc một số kiểu DGA botnet nhất định.…”

Section: Giới Thiệuunclassified

“…Thứ nhất là chúng tôi sử dụng thêm các đặc trưng thống kê từ tên miền đầu vào. Các công trình nghiên cứu [6,[9][10][11][12] đều đề cập các đặc trưng này và đã chứng minh tính hiệu quả của chúng trong phát hiện một số dạng DGA botnet nhất định. Vì vậy, đặc trưng thống kê có thể được sử dụng kết hợp nhằm nâng cao tỷ lệ phát hiện đúng của mạng LSTM truyền thống.…”

Section: Giới Thiệuunclassified

Phương pháp cải tiến LSTM dựa trên đặc trưng thống kê trong phát hiện DGA botnet

Hiếu¹,

Vạn²,

Tùng³

et al. 2018

Res. Dev. Info. Commun. Tech. J.

View full text Add to dashboard Cite

Phần lớn botnet sử dụng cơ chế sinh tên miền tự động (DGA: Domain Generation Algorithms) để kết nối và nhận lệnh từ máy chủ điều khiển. Việc tìm ra dạng DGA botnet thực hiện qua xác định cách thức tạo sinh tên miền đặc trưng cho loại botnet đó dựa trên những phân tích đặc trưng tên miền thu thập từ các truy vấn DNS. Trong bài báo này chúng tôi đề xuất phương pháp phân tích tên miền và phát hiện DGA botnet dựa trên sự kết hợp mạng LSTM (Long Short-Term Memory) với các đặc trưng thống kê như độ dài, entropy, mức độ ý nghĩa của tên miền nhằm tăng khả năng khái quát hóa cho mạng LSTM. Phương pháp đề xuất được thử nghiệm và đánh giá trên bộ dữ liệu tên miền thu thập trong thực tế bao gồm một triệu tên miền Alexa và hơn 750 nghìn tên miền được sinh bởi 37 loại DGA botnet. Kết quả thử nghiệm đã chứng minh tính hiệu quả của phương pháp đề xuất trong cả hai trường hợp phân loại hai lớp và phân loại đa lớp, với giá trị macro-averaging F1-score cao hơn 5% và nhận biết thêm được 3 loại DGA so với phương pháp phát hiện DGA botnet dựa trên mạng LSTM truyền thống. DOI: 10.32913/rd-ict.vol3.no40.528

show abstract

DGA Botnet detection using Collaborative Filtering and Density-based Clustering

Cited by 18 publications

References 4 publications

Detecting Word-Based Algorithmically Generated Domains Using Semantic Analysis

Detecting Word-Based Algorithmically Generated Domains Using Semantic Analysis

Recommendation and Classification Systems: A Systematic Mapping Study

Phương pháp cải tiến LSTM dựa trên đặc trưng thống kê trong phát hiện DGA botnet

Contact Info

Product

Resources

About