Characterizing the Behavior of a Program Using Multiple-Length N-Grams

Abstract

Some recent advances in intrusion detection are based on detecting anomalies in program behavior, as characterized by the sequence of kernel calls the program makes. Specifically, traces of kernel calls are collected during a training period. The substrings of fixed length N (for some N) of those traces are called N-grams. The set of N-grams occurring during normal execution has been found to discriminate effectively between normal behavior of a program and the behavior of the program under attack. The N-gram characterization, while effective, requires the user to choose a suitable value for N. This paper presents an alternative characterization, as a finite state machine whose states represent predictive sequences of different lengths. An algorithm is presented to construct the finite state machine from training data, based on traditional string-processing data structures but employing some novel techniques.

User profiling in intrusion detection: A review

“…Their results showed that their method could find about 75% of anomalous system calls. Marceau [15] and Cabrera et al [55] undertook similar experiments using different methods. The former uses multiple length n-grams on system logs and shows a high rate (13/20) of finding anomalies with fewer datasets than Forrest and Longstaff, and by analysing the normal dictionary, which stores the permitted sequences of system calls, shows that it was possible to detect anomalies with a detection rate of 75-100%.…”

“…Instead it monitors network traffic and compares it against an established baseline, where the baseline identifies what is "normal" for that network, what protocols are generally used, what ports and devices generally connect to each other. It alerts the administrator or user when anomalous or significantly different traffic is detected [13][14][15][16][17]. However, it may miss both known and novel attacks if they are not manifested along the observed dimensions.…”

Language models for detection of unknown attacks in network traffic

“…In the ensuing work these ideas were extended through application of Hidden Markov Models [21], feed-forward and recursive neural networks [23], rule induction algorithms [65] and Support Vector Machines [9]. As part of this evolution, trie and suffix tree data structures were introduced for storage and analysis of system call n-grams [22,24,66]. Beside system call analysis, n-gram models have recently been applied as part of host-based intrusion detection for identification of malicious code in program binaries and documents [e.g.…”

“…N-grams have been extensively used in host-based intrusion detection for modeling traces of system calls [e.g. [19][20][21][22][23][24], but until recently have not been applied in the context of network intrusion detection for n > 1. Tokenized words have been used for anomaly detection using rule-based learning [e.g.…”

Physically locating wireless intruders

“…The WIDS APs will detect intrusions based on both known attack signatures (such as network probes from NetStumbler or other non-passive "war-driving" programs ) and behavior based signatures (such as an internal MAC address suddenly appearing in an external location on a machine with different OS characteristics than it had previously). WIDS will detect anomalous behavior by tying the signature data into a behavior-based intrusion detection system [6,9].…”