Contextual multi-armed bandits for web server defense

Abstract

Abstract-In this paper we argue that contextual multi-armed bandit algorithms could open avenues for designing self-learning security modules for computer networks and related tasks. The paper has two contributions: a conceptual and an algorithmical one. The conceptual contribution is to formulate the real-world problem of preventing HTTP-based attacks on web servers as a one-shot sequential learning problem, namely as a contextual multi-armed bandit. Our second contribution is to present CMABFAS, a new and computationally very cheap algorithm for general contextual multi-armed bandit learning that specifically targets domains with finite actions. We illustrate how CMABFAS could be used to design a fully self-learning meta filter for web servers that does not rely on feedback from the end-user (i.e., does not require labeled data) and report first convincing simulation results.

Contextual Multi-armed Bandits under Feature Uncertainty

“…The learner often can access to contextual information in addition to rewards and selected actions, which are referred as contextual MAB Langford & Zhang (2008). Examples include personalized recommendation Bouneffouf et al (2012), web server defense Jung et al (2012) and information retrieval Hofmann et al (2011). For instance, the learner see feature vectors z 1 (t), z 2 (t), .…”