Learning DFA representations of HTTP for protecting web applications

Abstract

Intrusion detection is a key technology for self-healing systems designed to prevent or manage damage caused by security threats. Protecting web server-based applications using intrusion detection is challenging, especially when autonomy is required (i.e., without signature updates or extensive administrative overhead). Web applications are difficult to protect because they are large, complex, highly customized, and often created by programmers with little security background. Anomaly-based intrusion detection has been proposed as a strategy to meet these requirements. This paper describes how DFA (Deterministic Finite Automata) induction can be used to detect malicious web requests. The method is used in combination with rules for reducing variability among requests and heuristics for filtering and grouping anomalies. With this setup a wide variety of attacks is detectable with few false-positives, even when the system is trained on data containing benign attacks (e.g., attacks that fail against properly patched servers).

Comparing Anomaly Detection Techniques for HTTP

“…For some of the six algorithms, we also consider them in isolation. We also implemented the character distribution metric described by Wang and Stolfo [23], and the DFA induction and n-grams described by Ingham et al and Ingham [33,32].…”

“…The algorithm does not require negative examples. This algorithm is described in detail by Ingham et al [33].…”

“…The data types that are checked for valid form are host names, IP addresses, dates, various hash values (PHP session IDs, HTTP entity tags, etc), floating point numbers (HTTP q-values), and email addresses. Ingham and Ingham et al provide a detailed description of these heuristics in [32,33].…”

“…Generalization occurs in the DFA generation described in [33]. It also occurs when one or more "missed tokens" are allowed.…”

Boosting Web Intrusion Detection Systems by Inferring Positive Signatures

“…Ingham et al [22] use a deterministic finite automaton (DFA) to build a profile of legal HTTP requests. It works by tokenizing HTTP request parameters, and storing each token type and (optionally) its value.…”