Dataflow anomaly detection

Abstract

Beginning with the work of Forrest et al, several researchers have developed intrusion detection techniques based on modeling program behaviors in terms of system calls. A weakness of these techniques is that they focus on control flows involving system calls, but not their arguments. This weakness makes them susceptible to several classes of attacks, including attacks on security-critical data, race-condition and symbolic link attacks, and mimicry attacks. To address this weakness, we develop a new approach for learning dataflow behaviors of programs. The novelty in our approach, as compared to previous system-call argument learning techniques, is that it learns temporal properties involving the arguments of different system calls, thus capturing the flow of security-sensitive data through the program. An interesting aspect of our technique is that it can be uniformly layered on top of most existing control-flow models, and can leverage control-flow contexts to significantly increase the precision of dataflows captured by the model. This contrasts with previous system-call argument learning techniques that did not leverage control-flow information, and moreover, were focused on learning statistical properties of individual system call arguments. Through experiments, we show that temporal properties enable detection of many attacks that aren't detected by previous approaches. Moreover, they support formal reasoning about security assurances that can be provided when a program follows its dataflow behavior model, e.g., tar would read only files located within a directory specified as a command-line argument.

Anomalous Taint Detection

“…to be more precise of dynamic learning-based techniques, is that they are able to automatically infer security policies by observing and characterizing applications' events, as successfully shown in recent and past research efforts [37,35,9,15,29,21,20,2]. For instance, sequences of system calls can be used as a starting point to define a possible behavior of an application [15], statistical models can help to characterize system calls arguments usage [21,20], and machine learning techniques can be adopted to infer relationships among the arguments of different system calls [2].…”

“…As other approaches [2,29,21], our analysis is context-sensitive. That is, it considers contexts for each system call that can be utilized to refine argument learning.…”

“…In fact, the main goal of our anomalous taint detection approach is to couple taint analysis with anomaly detection so that each technique can improve its drawbacks by exploiting the advantages provided by the other. The approach, in fact, is general enough so that more powerful learning-rules, such as the one proposed in literature [20,2], can be plugged into it. Therefore, if we consider only FPs caused by violation to the taintedness of sinks' arguments, it is easy to see that this policy performed fairly well for ProFTPD (3 × 10 −5 FPs rate), while it performed well for Apache, by not reporting FPs at all.…”

“…Learning-based anomaly detection techniques [37,35,9,15,29,21,20,2] have been pursued for many years due to their ability to detect a broad range of attacks, including novel attacks. More recently, taint-tracking techniques (also known as information-flow techniques) [30,23,5,26,24,38] have become popular due to their high accuracy and the ability to detect a broad range of attacks.…”

“…For instance, sequences of system calls can be used as a starting point to define a possible behavior of an application [15], statistical models can help to characterize system calls arguments usage [21,20], and machine learning techniques can be adopted to infer relationships among the arguments of different system calls [2]. Unfortunately, all these techniques suffer of false positives (FPs) and attempts to limit them generally increase false negatives (FNs) as well.…”