A Comparative Study of Malicious URL Detection: Regular Expression Analysis, Machine Learning, and VirusTotal API

Abstract

In this paper, malicious URL detection was done in 3 ways. In the first method, self-written code was used that breaks up the URL into several fragments and then using functions and checkers analyses whether the URL is malicious. The second approach involved utilizing a fixed csv dataset containing website URLs from over 11,000 websites. Each entry in the dataset included 30 parameters describing the website and a class label indicating whether it was classified as a phishing website (1) or not (-1). 9 machine learning models were compared on this dataset to see which models gave the highest accuracy and F1-score. The last method was done using VirusTotal API Key. The user has to input the URL of a website and the code classifies it as malicious or not. A report is also generated if the URL is detected as malicious. The report is a compilation of the analysis of that website by several security vendors.