Partitioned Problem Space (PaPS) Ensemble For Zero-day Intrusion Detection

Abstract

The low cost and easy availability of cloud data storage have led to a massive increase in data being generated and stored for analytics, simultaneously amplifying challenges in data security. Detecting and stopping a data breach before it can cause any material damage and legal repercussions has become crucial. The volume and speed of data moving around the clouds make traditional intrusion detection systems ineffective. However, deep learning ensembles are a type of machine learning technique that combines the predictions of multiple deep learning models to produce a more accurate and reliable prediction. Particularly adept at identifying novel or sophisticated intrusions, these ensembles uncover patterns overlooked by individual models. In this work, we propose a novel partitioned problem space deep-learning ensemble approach wherein the base model’s problem space is constrained by limiting the number of classes and number of features visible to each during their training. The output probabilities of these base models are fed as input to the final meta-learner model along with a selective subset of important features. Our exper- imentation demonstrates that this ensemble approach outperforms known approaches in zero-day intrusion detection tasks. Our work aims to partition a complex problem space into smaller, more manageable sub-spaces. We propose a multi-stage deep learning ensemble model where base models are trained on a subset of classes and features, and their predictions are aggregated by a meta-learner model. This approach improves the model’s ability to generalize to unseen data, including zero-day attacks.