Towards the Detection of Malicious URL and Domain Names Using Machine Learning

Abstract

Malicious Uniform Resource Locator (URL) is an important problem in web search and mining. Malicious URLs host unsolicited content (spam, phishing, drive-by downloads, etc.) and try to lure uneducated users into clicking in such links or downloading malware which will result in critical data exfiltration. Traditional techniques in detecting such URLs have been to use blacklists and rule-based methods. The main disadvantage of such problems is that they are not resistant to 0-day attacks, meaning that there will be at least one victim for each URL before the blacklist is created. Other techniques include having sandbox and testing the URLs before clicking on them in the production or main environment. Such methods have two main drawbacks which are the cost of the sandboxing as well as the non-real-time response which is due to the approval process in the test environment. In this paper, we propose a method that exploits semantic features in both domains and URLs as well. The method is adaptive, meaning that the model can dynamically change based on the new feedback received on the 0-day attacks. We extract features from all sections of a URL separately. We then apply three methods of machine learning on three different sets of data. We provide an analysis of features on the most efficient value of N for applying the N-grams to the domain names. The result shows that Random Forest has the highest accuracy of over 96% and at the same time provides more interpretability as well as performance benefits.

Evaluating Supervised Machine Learning Models for Zero-Day Phishing Attack Detection: A Comprehensive Study

“…Another research used semantic features from domains and URLs to detect malicious URLs (Ghalati et al, 2020). The authors introduced an adaptive method that can dynamically change based on new feedback received on 0-day attacks, and they found that Random Forest has the highest accuracy of over 96% with more interpretability and performance bene ts.…”

“…Also, six out of ten papers have used Neural Networks (NN) and Deep Learning in their models and only two of them applied non-NN classi cation models. In addition, except for one research(Abdelnabi et al, 2020) that used visual data in their model, other papers utilized URLs in their work, and some of them applied NLP-based feature extraction methods to build a structured data set(Ghalati et al, 2020, Sahingoz et al, 2019 model and two categories of features: lexical features extracted from the URL strings and tokens frequency using the TF-IDF algorithm. Experimental results showed that the system achieved a false negative rate of only 1.35%.…”

"VNeSafe: Machine Learning-Assisted System for Detecting Malicious URLs and Spam Calls"

“…The most common types of attacks that a user may encounter are malicious Uniform Resource Locators (URLs) and spam calls. It has been observed that 39% of URLs are malicious [2]. Malicious URLs can be used to instantiate drive-by-download, phishing, and spam.…”