Behavior-based modeling and its application to Email analysis

Abstract

The Email Mining Toolkit (EMT) is a data mining system that computes behavior profiles or models of user email accounts. These models may be used for a multitude of tasks including forensic analyses and detection tasks of value to law enforcement and intelligence agencies, as well for as other typical tasks such as virus and spam detection. To demonstrate the power of the methods, we focus on the application of these models to detect the early onset of a viral propagation without "content-based" (or signature-based) analysis in common use in virus scanners. We present several experiments using real email from 15 users with injected simulated viral emails and describe how the combination of different behavior models improves overall detection rates. The performance results vary depending upon parameter settings, approaching 99% true positive (TP) (percentage of viral emails caught) in general cases and with 0.38% false positive (FP) (percentage of emails with attachments that are mislabeled as viral). The models used for this study are based upon volume and velocity statistics of a user's email rate and an analysis of the user's (social) cliques revealed in the person's email behavior. We show by way of simulation that virus propagations are detectable since viruses may emit emails at rates different than human behavior suggests is normal, and email is directed to groups of recipients in ways that violate the users' typical communications with their social groups.

Workload Modeling for Computer Systems Performance Evaluation

“…• When a system is attacked its workload may be significantly different from the normal workload [96,664,708,117]. For example, this happens with distributed denial-of-service (DDoS) attacks, and may also happen when it is infected by a computer virus.…”

“…An alternative is to devise several independent tests for abnormality and use them together. In the context of virus spreading by email, these tests can include user cliques (the sets of users that typically communicate with each other), the relative popularity of different users (not all users receive the same amount of email), and the rates at which emails are sent (again different users have different profiles) [664]. If several of these change, virus activity may be suspected.…”

“…Given a log of activity on a server, large outbreaks may indicate automated attacks. Identifying them and separating them from the general workload model then achieves two objectives: it prevents optimization of the server to handle attack loads, and it enables the development of tools that will identify and counteract attacks [664].…”

“…However, there is little data about its composition, except the obvious characteristics of bulk email: huge numbers of copies of the same messages. As for viruses that spread by email, analysis of user behavioral patterns (and especially deviations from them) has been suggested as an effective identification method, independent of the actual mail contents [664].…”

A Security Model for Detecting Suspicious Patterns in Physical Environment

“…Intrusion Detection System (IDS) is a mature technology that detects intrusion by monitoring activities in several aspects of the network or operating systems, piecing scattered information together for some insights [1]. Likewise, emails could be traced and related to the user profiles for modeling behavior [2]. A lot of research effort has been devoted to intrusion detection on virtual platform.…”