A Hidden Attack Sequences Detection Method Based on Dynamic Reward Deep Deterministic Policy Gradient

Liu, Yi; Pan, Zhisong; Pan, Yu; Guo, Shize; Liu, Yi; Xia, Shiming; Zheng, Qibin; Li, Hongmei; Bai, Wei

doi:10.1155/2022/1488344

Cited by 4 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An upgrade of DDPG is accomplished as dynamic reward DDPG in [55] which shows 97.46% accuracy in detecting attackers. The authors of [56] propose a DDPG IDS approach to achieve a detection accuracy of 97.28% in the WUSTIL-IIOT-2021 test set.…”

Section: Rl Policy Gradient Algorithms and Its Variantsmentioning

confidence: 99%

Untitled

2023

IJCNC

View full text Add to dashboard Cite

Addressing real-time network security issues is paramount due to the rapidly expanding IoT jargon. The erratic rise in usage of inadequately secured IoT-based sensory devices like wearables of mobile users, autonomous vehicles, smartphones and appliances by a larger user community is fuelling the need for a trustable, super-performant security framework. An efficient anomaly detection system would aim to address the anomaly detection problem by devising a competent attack detection model. This paper delves into the Deep Deterministic Policy Gradient (DDPG) approach, a promising Reinforcement Learning platform to combat noisy sensor samples which are instigated by alarming network attacks. The authors propose an enhanced DDPG approach based on trust metrics and belief networks, referred to as Deep Deterministic Policy Gradient Belief Network (DDPG-BN). This deep-learning-based approach is projected as an algorithm to provide "Deep-Defense" to the plethora of network attacks. Confidence interval is chosen as the trust metric to decide on the termination of sensor sample collection. Once an enlisted attack is detected, the collection of samples from the particular sensor will automatically cease. The evaluations and results of the experiments highlight a better detection accuracy of 98.37% compared to its counterpart conventional DDPG implementation of 97.46%. The paper also covers the work based on a contemporary Deep Reinforcement Learning (DRL) algorithm, the Actor Critic (AC). The proposed deep learning binary classification model is validated using the NSL-KDD dataset and the performance is compared to a few deep learning implementations as well.

show abstract

Section: Rl Policy Gradient Algorithms and Its Variantsmentioning

confidence: 99%

Untitled

2023

IJCNC

View full text Add to dashboard Cite

show abstract

“…Attacks from outside the company (external intrusions) and internal intrusions are among the security lapses. In recent few research, DRL has been used to defend systems against network intrusion attacks and solve the problem [51,58].…”

Section: ) Cyber Attack Intrusion Detectionmentioning

confidence: 99%

Deep Reinforcement Learning for Anomaly Detection: A Systematic Review

et al. 2022

View full text Add to dashboard Cite

Anomaly detection has been used to detect and analyze anomalous elements from data for years. Various techniques have been developed to detect anomalies. However, the most convenient one is Machine learning which is performing well but still has limitations for large-scale unlabeled datasets. Deep Reinforcement Learning (DRL) based techniques outperform the existing supervised or unsupervised and other alternative techniques for anomaly detection. This study presents a Systematic Literature Review (SLR), which analyzes DRL models that detect anomalies in their application. This SLR aims to analyze the DRL frameworks for anomaly detection applications, proposed DRL methods, and their performance comparisons against alternative methods. In this review, we have identified 32 research articles from 2017-2022 that discuss DRL techniques for various anomaly detection applications. After analyzing the selected research articles, this paper presents 13 different applications of anomaly detection found in the selected research articles. We identified 50 different datasets applied in experiments on anomaly detection and demonstrated 17 distinct DRL models used in the selected papers to detect anomalies. Finally, we analyzed the performance of these DRL models and reviewed them. Additionally, we observed that detecting anomalies using DRL frameworks is a promising area of research and showed that DRL had shown better performance for anomaly detection where other models lack. Therefore, we provide researchers with recommendations and guidelines based on this review.

show abstract

“…In the structure of actor-critic, the update of the actor policy depends on the critic value function [31][32][33]. Given the online network parameter φ, ϕ approx denotes the updated parameter of the actor network calculated by the estimated maximal value function max (Q θ (s, a)), ϕ true denotes the parameter obtained by using the actual value function Q π (s, a), where Q π (s, a) is unknown in the training process which represents the value function in an ideal state, then ϕ approx and ϕ true can be expressed in the following equation:…”

Section: Error Analysis It Is An Inevitable Problem For Q-mentioning

confidence: 99%

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

Zhang¹,

Zhang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The traditional Deep Deterministic Policy Gradient (DDPG) algorithm has been widely used in continuous action spaces, but it still suffers from the problems of easily falling into local optima and large error fluctuations. Aiming at these deficiencies, this paper proposes a dual-actor-dual-critic DDPG algorithm (DN-DDPG). First, on the basis of the original actor-critic network architecture of the algorithm, a critic network is added to assist the training, and the smallest Q value of the two critic networks is taken as the estimated value of the action in each update. Reduce the probability of local optimal phenomenon; then, introduce the idea of dual-actor network to alleviate the underestimation of value generated by dual-evaluator network, and select the action with the greatest value in the two-actor networks to update to stabilize the training of the algorithm process. Finally, the improved method is validated on four continuous action tasks provided by MuJoCo, and the results show that the improved method can reduce the fluctuation range of error and improve the cumulative return compared with the classical algorithm.

show abstract

A Hidden Attack Sequences Detection Method Based on Dynamic Reward Deep Deterministic Policy Gradient

Cited by 4 publications

References 24 publications

Untitled

Untitled

Deep Reinforcement Learning for Anomaly Detection: A Systematic Review

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

Contact Info

Product

Resources

About