Deep Reinforcement Learning for DER Cyber-Attack Mitigation

Roberts, Ciaran; Ngo, Sy-Toan; Milesi, Alexandre; Peisert, Sean; Arnold, Dan; Saha, Shammya Shananda; Scaglione, Anna; Johnson, Nathan G.; Kocheturov, Anton; Fradkin, Dmitriy

doi:10.1109/smartgridcomm47815.2020.9302997

Cited by 19 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their scheme detects the voltage variances based on historical data and auto-adjusts the voltage of connected nodes to mitigate any malicious activity. Based on a similar technique of measuring the voltage variances, Ciaran et al [22] proposed a tool to predict future attacks on a node by comparing its performance statistics. In this model, the selected node uses deep reinforcement learning and its parameters are stored.…”

Section: Related Workmentioning

confidence: 99%

“…Vincenzo's [15] and Ashutosh's [16] frameworks are performance friendly but these may not detect the hidden vulnerability as these models are based on a reactive approach rather than a proactive approach; where early detection of vulnerabilities is essential for securing the system. Likewise, in [20][21][22], IoT-and ICS-related threat detection and mitigation solutions were proposed to address specific network topologies. These solutions may not be extended to all types of nodes in hybrid networks.…”

Section: Gap Statementsmentioning

confidence: 99%

See 1 more Smart Citation

Sensitivity Analysis for Vulnerability Mitigation in Hybrid Networks

2022

View full text Add to dashboard Cite

The development of cyber-assured systems is a challenging task, particularly due to the cost and complexities associated with the modern hybrid networks architectures, as well as the recent advancements in cloud computing. For this reason, the early detection of vulnerabilities and threat strategies are vital for minimising the risks for enterprise networks configured with a variety of node types, which are called hybrid networks. Existing vulnerability assessment techniques are unable to exhaustively analyse all vulnerabilities in modern dynamic IT networks, which utilise a wide range of IoT and industrial control devices (ICS). This could lead to having a less optimal risk evaluation. In this paper, we present a novel framework to analyse the mitigation strategies for a variety of nodes, including traditional IT systems and their dependability on IoT devices, as well as industrial control systems. The framework adopts avoid, reduce, and manage as its core principles in characterising mitigation strategies. Our results confirmed the effectiveness of our mitigation strategy framework, which took node types, their criticality, and the network topology into account. Our results showed that our proposed framework was highly effective at reducing the risks in dynamic and resource constraint environments, in contrast to the existing techniques in the literature.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Gap Statementsmentioning

confidence: 99%

Sensitivity Analysis for Vulnerability Mitigation in Hybrid Networks

2022

View full text Add to dashboard Cite

show abstract

“…As shown in Fig. 1, in this paper, instead of controlling arbitrarily the parameters of the VV/VW curves, we only shift the value from the default value defining the curves [6,24]. The vector of actions a, output of the NN that approximates the optimum policy, is a function of the three-phase voltages at all, or part, of buses in the distribution system, which is the observation/input of the STGCN-DRL.…”

Section: B Drl Design For Vvcmentioning

confidence: 99%

“…The cyber attack is launched by translating the VV/VW curves and shifting η of the compromised smart inverter to induce an oscillation (the setup is borrowed from [6]). In Fig.…”

Section: B Full Observationsmentioning

confidence: 99%

“…In distribution systems, voltage profiles are the most critical indicator of the system operating condition, whilst reliable and efficient energy management is the core task [1][2][3][4]. This is why Volt-VAR control (VVC) schemes have been developed and integrated into distribution systems to reduce network losses [2], avoid voltage violations [5] and mitigate cyber attacks [6]. However, the rapid growth of distributed energy resources makes it increasingly difficult to manage voltage profiles on active distribution networks.…”

Section: Introduction a Background And Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning with Graph ConvNets for Distribution Network Voltage Control

Wu¹,

Carreño²,

Scaglione³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a model-free Volt-VAR control (VVC) algorithm via the spatio-temporal graph ConvNet-based deep reinforcement learning (STGCN-DRL) framework, whose goal is to control smart inverters in an unbalanced distribution system. We first identify the graph shift operator (GSO) based on the power flow equations. Then, we develop a spatiotemporal graph ConvNet (STGCN), testing both recurrent graph ConvNets (RGCN) and convolutional graph ConvNets (CGCN) architectures, aimed at capturing the spatiotemporal correlation of voltage phasors. The STGCN layer performs the feature extraction task for the policy function and the value function of the reinforcement learning architecture, and then we utilize the proximal policy optimization (PPO) to search the action spaces for an optimum policy function and to approximate an optimum value function. We further utilize the low-pass property of voltage graph signal to introduce an GCN architecture for the the policy whose input is a decimated state vector, i.e. a partial observation. Case studies on the unbalanced 123-bus systems validate the excellent performance of the proposed method in mitigating instabilities and maintaining nodal voltage profiles within a desirable range.

show abstract

Reinforcement Learning using Physics Inspired Graph Convolutional Neural Networks

Scaglione

Arnold

2022

2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

Deep Reinforcement Learning for DER Cyber-Attack Mitigation

Cited by 19 publications

References 8 publications

Sensitivity Analysis for Vulnerability Mitigation in Hybrid Networks

Sensitivity Analysis for Vulnerability Mitigation in Hybrid Networks

Deep Reinforcement Learning with Graph ConvNets for Distribution Network Voltage Control

Reinforcement Learning using Physics Inspired Graph Convolutional Neural Networks

Contact Info

Product

Resources

About