Multi-Agent Deep Reinforcement Learning for Sectional AGC Dispatch

Li, Jiawen; Yu, Tao; Zhu, Hanxin; Li, Fusheng; Lin, Dan; Li, Zhuohuan

doi:10.1109/access.2020.3019929

Cited by 14 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The agent applies the methods described in algorithms ( 6) and (7) to read the labeled attacks and predict the targeted attack labels. Based on the results, our DRL model effectively classifies threats in real time and provides detection and response [29]. C.) Lastly, we offer valuable insights for future research directions regarding the use of DRL to detect cybersecurity attacks in the SCADA domain.…”

Section: B Contributionsmentioning

confidence: 89%

“…Lastly, we trained and evaluated our model using the selected datasets: WUSTL-IIoT-2018 [27] and WUSTL-IIoT-2021 [28], comprised of network traffic protocols. After successfully training the model, our DRL learns to classify threats in real-time and provides detection and response [29]. In retrospect to validation V-C2, we loaded the saved model with our test data, comprised of multiple binary classification inputs.…”

Section: ) Performance Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure

Mesadieu,

Torre,

Chennameneni

2024

IEEE Access

View full text Add to dashboard Cite

The prevalence of cyber-attacks perpetrated over the last two decades, including coordinated attempts to breach targeted organizations, has drastically and systematically exposed some of the more critical vulnerabilities existing in our cyber ecosystem. Particularly in Supervisory Control and Data Acquisition (SCADA) systems with targeted attacks aiming to bypass signature-based protocols, attempting to gain control over operational processes. In the past, researchers utilized deep learning and reinforcement learning algorithms to mitigate threats against industrial control systems (ICS). However, as technology evolves, these techniques become ineffective in monitoring and enhancing the cybersecurity defenses of those system against unwanted attacks. To address these concerns, we propose a deep reinforcement learning (DRL) framework for anomaly detection in the SCADA network. Our model utilizes a "Q-network", which allows it to achieve state-of-the-art performance in pattern recognition from complex tasks. We validated our solution on two publicly available datasets. The WUSTL-IIoT-2018 and the WUSTL-IIoT-2021, each comprised of twenty-five networking features representing benign and attack traffic. The results showed that our model successfully achieved an accuracy of 99.36% in attack detection, highlighting DRL's potential to enhance the security of critical infrastructure and laying the foundation for future research in this domain.

show abstract

Section: B Contributionsmentioning

confidence: 89%

Section: ) Performance Metricsmentioning

confidence: 99%

Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure

Mesadieu,

Torre,

Chennameneni

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Compared with control battery energy storage system (BESS), the reinforcement learning is more economical and effective in reducing the voltage problems. Reference [32], proposed an Automatic Generation Control (AGC) control method based on multi-agent reinforcement learning, which takes system economy and security into account. Latterly, the reinforcement learning was used to construct an online DSTATCOM control strategy to improve power quality [33].…”

Section: B Study Of Systematic Methods To Tnepmentioning

confidence: 99%

Transmission Network Dynamic Planning Based on a Double Deep-Q Network With Deep ResNet

Wang

Zhou

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Based on a Double Deep-Q Network with deep ResNet (DDQN-ResNet), this paper proposes a novel method for transmission network expansion planning (TNEP). Since TNEP is a large scale and mixedinteger linear programming (MILP) problem, as the transmission network scale and the optimal constraints increase, the numerical calculation and heuristic learning-based methods suffer from heavy computational complexities in calculation and training. Besides, due to the black box characteristic, the solution processes of the heuristic learning-based methods are inexplicable and usually require repeated training. By using DDQN-ResNet, this paper constructs a high-performance and flexible method to solve large-scale and complex-constrained TNEP problem. Firstly, we form a two-objective TNEP model, in which one objective is to minimize the comprehensive cost, and another objective is to maximize the transmission network reliability. The comprehensive cost takes into account the expansion cost, the network loss cost, and the maintenance cost. The transmission network reliability is evaluated by the expected energy not served (EENS) and the electrical betweenness. Secondly, from the TNEP model, the TNEP task is constructed based on the Markov decision process. By abstracting the task, the TNEP environment is obtained for DDQN-ResNet. In addition, to identify the construction value of lines, an agent is establish based on DDQN-ResNet. Finally, we perform the static planning and visualize the reinforcement learning process. The dynamic planning is realized by reusing the training experience. The validity and flexibility of DDQN-ResNet are verified on RTS 24-bus test system.

show abstract

“…In addition to this, a similar study is the deep policy dynamics based win or learn fast-policy hill climbing network [85] for power grids containing multiple renewable energy sources. In addition, there are AGC studies in conjunction with scheduling, Li et al designed a segmented AGC scheduling framework using a hierarchical multi-agent deep deterministic policy gradient to fully consider security and economy in the scheduling process [86]. AGC control strategies using deep reinforcement learning are studied in a variety of ways in renewable energy-containing power grids, but deep reinforcement learning requires a high amount of data and the model is more specific and can only be used for a specific situation the system, and attention should be paid to this aspect of the enhancement in the use of the system.…”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

A review of control strategies for automatic generation control in power systems with renewable energy

Kuang,

Tian,

Liu

et al. 2024

Prog. Energy

View full text Add to dashboard Cite

This review presents a state-of-the-art literature review of Automatic Generation Control (AGC) control strategies for power systems containing renewable energy sources. The incorporation of renewable energy into the power system has a large impact on the stability, reliability, economy and security of the power system. To mitigate these effects, it is important to choose a suitable control strategy for AGC. However, there is a limited amount of literature available on the review of AGC in renewable energy power systems., so a review of AGC control strategies for renewable energy-containing power systems is necessary. The investigation of this paper focuses on all kinds of different AGC control strategies for renewable energy-containing power systems, such as Proportional Integral Derivative (PID) control, fuzzy control, artificial neural network control, etc., and compares and considers these different control methods, while this paper summarises the power system models with/without renewable energy. In addition, this paper summarises and discusses the application of intelligent optimization algorithms and energy storage systems to control strategies. The problems and future research directions of the current research on power systems with renewable energy sources are also discussed.

show abstract

Multi-Agent Deep Reinforcement Learning for Sectional AGC Dispatch

Cited by 14 publications

References 19 publications

Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure

Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure

Transmission Network Dynamic Planning Based on a Double Deep-Q Network With Deep ResNet

A review of control strategies for automatic generation control in power systems with renewable energy

Contact Info

Product

Resources

About