How Much Training Data is Enough? A Case Study for HTTP Anomaly-Based Intrusion Detection

Abstract

Most anomaly-based intrusion detectors rely on models that learn from training datasets whose quality is crucial in their performance. Albeit the properties of suitable datasets have been formulated, the influence of the dataset size on the performance of the anomaly-based detector has received scarce attention so far. In this work, we investigate the optimal size of a training dataset. This size should be large enough so that training data is representative of normal behavior, but after that point, collecting more data may result in unnecessary waste of time and computational resources, not to mention an increased risk of overtraining. In this spirit, we provide a method to find out when the amount of data collected at the production environment is representative of normal behavior in the context of a detector of HTTP URI attacks based on 1-grammar. Our approach is founded on a set of indicators related to the statistical properties of the data. These indicators are periodically calculated during data collection, producing time series that stabilize when more training data is not expected to translate to better system performance, which indicates that data collection can be stopped. We present a case study with real-life datasets collected at the University of Seville (Spain) and a public dataset from the University of Saskatchewan. The application of our method to these datasets showed that more than 42% of one trace, and almost 20% of another were unnecessarily collected, thereby showing that our proposed method can be an efficient approach for collecting training data at the production environment. INDEX TERMS Anomaly-based intrusion detection, dataset assessment, training.

On the Detection Capabilities of Signature-Based Intrusion Detection Systems in the Context of Web Attacks

“…A prerequisite is to collect enough volume of events from the operational environment [37]. The size of the collected dataset will have implications in the time spent in the tuning process since we will need to manually review some of the alerts generated.…”

“…In a way, this can be viewed as a type of voting scheme. In a previous work [37], we combined ModSecurity [38] and Snort [39] to sanitize HTTP traces by combining the output of both detectors (∪ operation). Sanitization, however, is meant to maximize TP but not to minimize FP.…”

Reputation-based Security model for detecting biased attacks in BigData

“…Using the sensors [2] and different IoT devices. Moreover, the security in big data is very challenging as it is concerned with attacks [3] that can originate either from online or offline spheres, hence, we can collect data and store the data using different protocols such as hypertext transfer protocol (HTTP) [4], message queue telemetry transport protocol (MQTT) [5], and constrained application protocol (CoAP) [6]. The data is stored so it can be utilized for the improvement of the device.…”