Robust Regional Coordination of Inverter-Based Volt/Var Control via Multi-Agent Deep Reinforcement Learning

Liu, Hangyue; Zhang, Cuo; Chai, Qingmian; Meng, Ke; Guo, Qinglai; Zhang, Dong

doi:10.1109/tsg.2021.3104139

Cited by 39 publications

(17 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large number of precision equipment with CPU, PLC, and other cores, these modern precision instruments, production equipment, etc. have very high requirements for power quality [1], sometimes just a momentary voltage dip may lead to very serious consequences, such as causing interruptions in the production process or equipment failure, thus causing enterprises to su er huge economic losses, as shown in Figure 1 for a variety of high-end manufacturing equipment characteristics and the impact of power quality [2,3]. ese power quality problems can directly or indirectly cause serious economic losses to the national economy and seriously a ect the safe, stable, and e cient operation of electric power.…”

Section: Introductionmentioning

confidence: 99%

Optimization of Voltage Dynamic Performance at Inverter Output with Machine Learning and Intelligent Virtual Impedance

Wang

2022

Mobile Information Systems

View full text Add to dashboard Cite

With the continuous improvement of industrial production and manufacturing level, all kinds of equipment are developing in the direction of refinement and precision, especially the equipment manufacturing industry such as automobile manufacturing, equipment, integrated circuits, chip manufacturing, and other semiconductor industries. An intelligent virtual resistance control strategy based on machine research is proposed for rational power distribution and cycle suppression when multiple inverters are operated in parallel in low-voltage microgrids. Based on this, the control strategy of temporary power reduction of inverters is improved to improve the parallel inverter power distribution accuracy and ensure the reliable dynamic response of the constrained current in the circuit between parallel inverters. Finally, a multi-inverter parallel system model is established to verify the correctness and effectiveness of the proposed control strategy. The voltage waveform under nonlinear load conditions is improved and high-frequency harmonics are suppressed, which has obvious advantages compared with the general virtual impedance design method.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimization of Voltage Dynamic Performance at Inverter Output with Machine Learning and Intelligent Virtual Impedance

Wang

2022

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

“…is the reactive power outputs of all controllable IB-ERs and SVCs at time t. Since the paper focus on the IB-VVC, for simply the problem, we assume the incidence matrix A is not changed in the process, so the information of A is not added into the state [1], [20].…”

Section: B Formulating the Ib-vvc As A One-step Markov Decision Processmentioning

confidence: 99%

“…To tackle the problem, Volt-Var control (VVC) is integrated into ADNs to minimize power loss and eliminate voltage violations by controlling the reactive power resources. Since most DGs are inverter-based energy resources (IB-ERs) that are possible to provide reactive power support rapidly, it is of crucial importance to utilize these resources to achieve the VVC [1], [2].…”

Section: Introductionmentioning

confidence: 99%

Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control

Liu¹,

Ye²,

Deng³

et al. 2022

Preprint

View full text Add to dashboard Cite

A one-step two-critic deep reinforcement learning (OSTC-DRL) approach for inverter-based volt-var control (IB-VVC) in active distribution networks is proposed in this paper. Firstly, considering IB-VVC can be formulated as a singleperiod optimization problem, we formulate the IB-VVC as a onestep Markov decision process rather than the standard Markov decision process, which simplifies the DRL learning task. Then we design the one-step actor-critic DRL scheme which is a simplified version of recent DRL algorithms, and it avoids the issue of Q value overestimation successfully. Furthermore, considering two objectives of VVC: minimizing power loss and eliminating voltage violation, we utilize two critics to approximate the rewards of two objectives separately. It simplifies the approximation tasks of each critic, and avoids the interaction effect between two objectives in the learning process of critic. The OSTC-DRL approach integrates the one-step actor-critic DRL scheme and the two-critic technology. Based on the OSTC-DRL, we design two centralized DRL algorithms. Further, we extend the OSTC-DRL to multi-agent OSTC-DRL for decentralized IB-VVC and design two multi-agent DRL algorithms. Simulations demonstrate that the proposed OSTC-DRL has a faster convergence rate and a better control performance, and the multi-agent OSTC-DRL works well for decentralized IB-VVC problems.

show abstract

“…Simultaneously, treating each distributed energy resource (DER) as an agent [9]- [11] poses scalability issues in ADNs that contain a large number of DERs. Reference [13] partitions the entire distribution network into multiple subnetworks and treats each of them as an agent to control internal DERs, thus significantly reducing the number of agents and promoting the scalability of the method. Reference [14] notes the unsatisfactory scalability of the existing DRL-based methods and proposes a scalable algorithm called distributed proximal policy optimization (DPPO) to address the issue of electric vehicle (EV) charging management.…”

Section: Introductionmentioning

confidence: 99%

Real-time Operation Optimization in Active Distribution Networks Based on Multi-agent Deep Reinforcement Learning

Xu,

Gao,

Wang

et al. 2024

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

The increasing integration of intermittent renewable energy sources (RESs) poses great challenges to active distribution networks (ADNs), such as frequent voltage fluctuations. This paper proposes a novel ADN strategy based on multiagent deep reinforcement learning (MADRL), which harnesses the regulating function of switch state transitions for the realtime voltage regulation and loss minimization. After deploying the calculated optimal switch topologies, the distribution network operator will dynamically adjust the distributed energy resources (DERs) to enhance the operation performance of ADNs based on the policies trained by the MADRL algorithm. Owing to the model-free characteristics and the generalization of deep reinforcement learning, the proposed strategy can still achieve optimization objectives even when applied to similar but unseen environments. Additionally, integrating parameter sharing (PS) and prioritized experience replay (PER) mechanisms substantially improves the strategic performance and scalability. This framework has been tested on modified IEEE 33-bus, IEEE 118bus, and three-phase unbalanced 123-bus systems. The results demonstrate the significant real-time regulation capabilities of the proposed strategy.

show abstract

Robust Regional Coordination of Inverter-Based Volt/Var Control via Multi-Agent Deep Reinforcement Learning

Cited by 39 publications

References 33 publications

Optimization of Voltage Dynamic Performance at Inverter Output with Machine Learning and Intelligent Virtual Impedance

Optimization of Voltage Dynamic Performance at Inverter Output with Machine Learning and Intelligent Virtual Impedance

Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control

Real-time Operation Optimization in Active Distribution Networks Based on Multi-agent Deep Reinforcement Learning

Contact Info

Product

Resources

About