GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learning

Chen, Jinyin; Shulong, Hu,; Zheng, Haibin; Xing, Changyou; Zhang, Guomin

doi:10.1016/j.cose.2022.103055

Cited by 22 publications

(11 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Attackers can make full use of artificial intelligence technologies such as reinforcement learning and imitation learning to make attack decisions and deployment, and then launch high-intensity, high-concealment, and high-destructive cyber attacks with automated and intelligent attack tools. Chen et al [ 19 ] introduce expert knowledge to guide the attack agent to make better decisions in RL-based automated attacks. Li et al [ 20 ] construct an improved network graph model and incorporates social engineering into automated penetration testing (PT) platforms.…”

Section: Related Workmentioning

confidence: 99%

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Yao

Wang

Xiong

et al. 2023

Entropy

View full text Add to dashboard Cite

Reinforcement learning has shown a great ability and has defeated human beings in the field of real-time strategy games. In recent years, reinforcement learning has been used in cyberspace to carry out automated and intelligent attacks. Traditional defense methods are not enough to deal with this problem, so it is necessary to design defense agents to counter intelligent attacks. The interaction between the attack agent and the defense agent can be modeled as a multi-agent Markov game. In this paper, an adversarial decision-making approach that combines the Bayesian Strong Stackelberg and the WoLF algorithms was proposed to obtain the equilibrium point of multi-agent Markov games. With this method, the defense agent can obtain the adversarial decision-making strategy as well as continuously adjust the strategy in cyberspace. As verified in experiments, the defense agent should attach importance to short-term rewards in the process of a real-time game between the attack agent and the defense agent. The proposed approach can obtain the largest rewards for defense agent compared with the classic Nash-Q and URS-Q algorithms. In addition, the proposed approach adjusts the action selection probability dynamically, so that the decision entropy of optimal action gradually decreases.

show abstract

Section: Related Workmentioning

confidence: 99%

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Yao

Wang

Xiong

et al. 2023

Entropy

View full text Add to dashboard Cite

show abstract

“…However, the CTF scenarios constructed for the experiments were only simplifed versions and were not experimented on relatively complex scenarios. Chen [12] frst proposed a generic intelligent PT framework based on GAIL. GAIL-PT addresses the problem of high labour costs due to the intervention of security experts and high-dimensional discrete action spaces.…”

Section: Use Of Expert Knowledgementioning

confidence: 99%

“…. S { } do (10) Sample action A from the behaviour policy (11) Te environment performs A and gives back R(reward), and the agent observes (S, A, R, S′) (12) Push the transition τ(S, A, R, S′) into D interact , overwriting oldest interaction transition if over capacity of D interact (13) Sample a batch size of k transitions from D replay with prioritization (14) Calculate loss J(Q) using the target network (15) Perform a gradient descent step to update the weights for the policy network θ (16) S′←S, the state transitions from S to S′ (17) end for (18) ifumodf f � 0thenθ′←θend if (19) of computer networks and cyber security concepts. We construct a simulated network scene through CBS and encapsulate this network scene into a gym environment where we can interact with an agent and further combine it with RL-related algorithms for our experiments.…”

Section: Main Experimental Proceduresmentioning

confidence: 99%

“…On the downside, its expert knowledge is limited by the constructed penetration test scenarios and is less interpretable and generalisable. Chen [12] proposed an intelligent PT framework named GAIL-PT. GAIL-PT collects expert knowledge via Metasploit and uses GAIL (generative adversarial imitation learning) in imitation learning for path planning.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DQfD-AIPT: An Intelligent Penetration Testing Framework Incorporating Expert Demonstration Data

Wang

Xiong

et al. 2023

Security and Communication Networks

View full text Add to dashboard Cite

The application of reinforcement learning (RL) methods of artificial intelligence for penetration testing (PT) provides a solution to the current problems of high labour costs and high reliance on expert knowledge for manual PT. In order to improve the efficiency of RL algorithms for PT, existing research has considered bringing in the knowledge of PT experts and combining it with the use of imitative learning methods to guide the agent in its decision-making. However, the disadvantage of using imitation learning is also obvious; that is, the performance of the strategies learned by the agent hardly exceeds the demonstrated behaviour of the expert and it can also cause expert knowledge overfitting. At the same time, the expert knowledge in the currently proposed method is poorly interpretable and highly scenario-dependent. The expert knowledge used in these methods is not universal. To address these issues, we propose an intelligent PT framework named DQfD-AIPT. The framework encompasses the process of collecting and using expert knowledge and provides a rational definition of the structure of expert knowledge. To solve the overfitting problem, we perform PT path planning based on the deep Q-learning from demonstrations (DQfD) algorithm. DQfD combines the benefits of RL and imitation learning to effectively improve the PT strategy and performance of agents while avoiding overfitting. Finally, we conducted experiments in a simulated network scenario containing honeypots. The experimental results proved the effectiveness of expert knowledge incorporation. In addition, the DQfD algorithm can improve the efficiency of penetration testing more effectively than that by the classical deep reinforcement learning (DRL) method and can obtain a higher cumulative reward. Not only that, due to the incorporation of expert knowledge, in scenarios with honeypots, the DQfD method can effectively reduce the probability of interacting with honeypots compared to the classical DRL method.

show abstract

“…This dynamic nature of the threat landscape presents a significant challenge to individuals, organizations, and institutions responsible for safeguarding digital assets and sensitive information. Cybercriminals exhibit remarkable adaptability, constantly refining their tactics, techniques, and procedures (TTPs) to stay one step ahead of security measures [5]. Just as security professionals identify and mitigate one vulnerability cybercriminals quickly shift their focus to discover new avenues of attack [6][7].…”

Section: Introductionmentioning

confidence: 99%

Penetration Testing Framework using the Q Learning Ensemble Deep CNN Discriminator Framework

Railkar,

Joshi

2024

IJACSA

View full text Add to dashboard Cite

Penetration testing (PT) serves as an effective tool for examining networks and identifying vulnerabilities by simulating a hacker's attack to uncover valuable information, such as details about the host's operating and database systems. Strong penetration testing is crucial for assessing system vulnerabilities in the constantly changing world of cyber security. Existing methods often struggle with adapting to dynamic threats, providing limited automation, and lacking the ability to discern subtle security weaknesses. In comparison to manual PT, intelligent PT has gained widespread popularity due to its efficiency, resulting in reduced time consumption and lower labor costs. Considering this, the effective penetration testing framework is developed using prairie natural swarm (PNS) optimized Q-learning ensemble deep CNN. Initially, the penetration testing environment (Shodan search engine) is simulated, and along with that expert knowledge base is also generated. Subsequently, the Nmap script engine and Metasploit are deployed, providing robust tools for network investigation and vulnerability assessment. The system state is then relayed to the Q-learning ensemble deep convolutional neural network (Qlearning ensemble deep CNN) classifier. This unique ensemble combines the strengths of Q-learning and deep CNNs, enabling optimal policy learning for decision-making. The prairie natural swarm optimization algorithm is developed through the hybridization of coyote and particle swarm characteristics to fine-tune classifier parameters, enhancing performance. Additionally, the discriminator is trained to maximize standard action rewards while minimizing discounted action rewards, distinguishing valuable from less valuable information. By evaluating the advantage function, successful penetration likelihood is determined, informing situational decision-making through the Q-learning ensemble deep CNN classifier. Accuracy, sensitivity, and specificity as well as the proposed PNS-optimized Q-learning ensemble deep model are used to evaluate the output. In comparison to other approaches currently in use, CNN achieves values of 94.54%, 94.40%, 94.90% for TP, 94.64%, 94.69%, and 94.52% for k-fold.

show abstract

GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learning

Cited by 22 publications

References 17 publications

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

DQfD-AIPT: An Intelligent Penetration Testing Framework Incorporating Expert Demonstration Data

Penetration Testing Framework using the Q Learning Ensemble Deep CNN Discriminator Framework

Contact Info

Product

Resources

About