Improving Knowledge Based Spam Detection Methods: The Effect of Malicious Related Features in Imbalance Data Distribution

Abstract

Spam is no longer just commercial unsolicited email messages that waste our time, it consumes network traffic and mail servers' storage. Furthermore, spam has become a major component of several attack vectors including attacks such as phishing, cross-site scripting, cross-site request forgery and malware infection. Statistics show that the amount of spam containing malicious contents increased compared to the one advertising legitimate products and services. In this paper, the issue of spam detection is investigated with the aim to develop an efficient method to identify spam email based on the analysis of the content of email messages. We identify a set of features that have a considerable number of malicious related features. Our goal is to study the effect of these features in helping the classical classifiers in identifying spam emails. To make the problem more challenging, we developed spam classification models based on imbalanced data where spam emails form the rare class with only 16.5% of the total emails. Different metrics were utilized in the evaluation of the developed models. Results show noticeable improvement of spam classification models when trained by dataset that includes malicious related features.

Incremental Learning for Spam Detection

“…Yang and Jungchen [9] presented "Uninterrupted Approaches for Spam Detection Based on SVM and AIS" which in eight methods have been studied and compared due to their speed and accuracy in spam detection. Alqatawna et al [10] presented an incremental learning algorithm based on group learning. Then a software of proposed algorithm for spam filtering has been discussed.…”

Voting-based Classification for E-mail Spam Detection

“…For example in Al-Jarrah, et al [6], the authors proposed a new set of features extracted from the headers of e-emails, which were then used for training common classifiers. In Alqatawna, et al [7], the authors focused on extracting malicious-related features and studied the effect of these features on the effectiveness of different classifiers. In Ruan and Tan [8], the authors proposed various approaches for constructing features for e-mail spam filtering (i.e.…”

“…is, am, are, etc.) X 4 Average word length X 5 Minimum word length X 6 Maximum word length X 7 Count of uppercase letters X 8 Count of lowercase letters X 9 Count of special characters X 10 Longest sequence of adjacent capital letters X 11 Count of spam words 1 X 12 Count of slang words 2 X 13 Count of semicolons X 14 Count of sentences (split by full stops) X 15 Count of alpha-numeric words X 16 Time units X 17 Links (i.e. number of tokens ending with {.net, .com, .jo, etc.}.…”

A new semantic attribute deep learning with a linguistic attribute hierarchy for spam detection

“…Wang et al [9] used four different types of feature sets (user features, content features, Uni-Bi features, and sentiment features) and their combinations to validate Random Forests for Twitter spam detection, and the experimental results show feature combination outperformed a single type of features on decision making. The study of Alqatawna et al [16] showed that adding malicious related features to training data significantly improved the detection of spam emails. Recently, He et al [17] investigated spam detection through analysing the features in email and message spams with a RBF-kernal SVM spam detector, thus to provide clues of spams to users.…”