Undermining an Anomaly-Based Intrusion Detection System Using Common Exploits

Abstract

Abstract. Over the past decade many anomaly-detection techniques have been proposed and/or deployed to provide early warnings of cyberattacks, particularly of those attacks involving masqueraders and novel methods. To date, however, there appears to be no study which has identified a systematic method that could be used by an attacker to undermine an anomaly-based intrusion detection system. This paper shows how an adversary can craft an offensive mechanism that renders an anomaly-based intrusion detector blind to the presence of on-going, common attacks. It presents a method that identifies the weaknesses of an anomaly-based intrusion detector, and shows how an attacker can manipulate common attacks to exploit those weaknesses. The paper explores the implications of this threat, and suggests possible improvements for existing and future anomaly-based intrusion detection systems.

NewApproach for Detecting Unknown Malicious Executables

“…Variety of anomaly detection methods utilizing this approach has been proposed [5][6][7][8][9][10][11]. Forrest [5], for example, introduced a simple anomaly detection method based on monitoring the system calls issued by privileged processes.…”

Automated Discovery of Mimicry Attacks

“…The seminal research on mimicry [9,24] and evasion attacks [20][21][22] demonstrated a critical shortcoming of model-based anomaly detection. Attackers can avoid detection by altering their attacks to appear as a program's normal execution.…”

“…Mimicry and evasion attacks avoid detection by transforming an attack sequence of system calls so that it is accepted by a program model yet still carries out the same malicious action. Previous research found examples of mimicry attacks against high-privilege processes restricted by a model-based detector [20][21][22]24]. However, the attacks were constructed manually by iterating between an attack sequence and a program model until the attack could be made to appear normal.…”

“…In experiments, we show that we can automatically discover the mimicry attack against the Stide [8] model for wu-ftpd [24] and the evasion attacks against the Stide models for passwd, restore, and traceroute [20][21][22]. The model checking process completed in about 2 seconds or less when undetected attacks were present in the models.…”

A framework for the evaluation of intrusion detection systems

“…Several researchers have pointed out the need to include the resistance against attacks as part of the evaluation of an IDS [25,27,11,34,29,30,13]. However, the traditional evaluation metrics are based on ideas mainly developed for nonsecurity related fields and therefore, they do not take into account the role of an adversary and the evaluation of the system against this adversary.…”