Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphs

Abstract

Reinforcement learning (RL), in conjunction with attack graphs and cyber terrain, are used to develop reward and state associated with determination of optimal paths for exfiltration of data in enterprise networks. This work builds on previous crown jewels (CJ) identification that focused on the target goal of computing optimal paths that adversaries may traverse toward compromising CJs or hosts within their proximity. This work inverts the previous CJ approach based on the assumption that data has been stolen and now must be quietly exfiltrated from the network. RL is utilized to support the development of a reward function based on the identification of those paths where adversaries desire reduced detection. Results demonstrate promising performance for a sizable network environment.

Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terrain

“…Another line of research focuses on developing more specific penetration testing tasks. A number of authors define more specific tasks by reward engineering and other modifications to the MDP including formulations of capture the flag [22], crown jewel analysis [16], and discovering exfiltration paths [17]. This paper extends this line of research with a methodology for exposing SDR.…”

“…Hu et al extend the use of the CVSS by proposing to use exploitability scores weight rewards [14]. Gangupantulu et al [15], [16] and Cody et al [17] explicitly extend the methods of Hu et al with concepts of terrain. Gangupantulu et al advocate defining models of terrain in terms of the rewards and transition probabilities of MDPs, first in the case of firewalls as obstacles [15], then in the case of lateral pivots nearby key terrain [16].…”

“…Gangupantulu et al advocate defining models of terrain in terms of the rewards and transition probabilities of MDPs, first in the case of firewalls as obstacles [15], then in the case of lateral pivots nearby key terrain [16]. Cody et al apply these concepts to exfiltration [17]. Other authors either handcraft the MDP or do not remark on how its components are estimated.…”

“…Gangupantulu et al [15] proposed that cyber terrain can be modeled into CVSS-MDPs by adding transition probabilities for traversing firewalls and negative rewards for different protocols. Cody et al [17] later modeled the services-based defensive terrain in CVSS-MDPs based on the assumption that the attackers can infer the presence of defenses terrain based on the services running on a host. We adopt their methods and classify the services into four categories: authentication, data, security and common.…”

“…The same network framework as in Cody et al [17] is used for our experiment but with different configurations in defense mechanism. To simulate real-world network conditions, there are layers of defenses between the Internet and the innermost private network.…”