Coordinated Optimization of Generation and Compensation to Enhance Short-Term Voltage Security of Power Systems Using Accelerated Multi-Objective Reinforcement Learning

Abstract

High proportions of asynchronous motors in demand-side have pressured heavily on short-term voltage security of receiving-end power systems. To enhance short-term voltage security, this paper coordinates the optimal outputs of generation and compensation in a multi-objective dynamic optimization model. With equipment dynamics, network load flows, lower and upper limitations, and security constraints considered, this model simultaneously minimizes two objectives: the expense of control decision and the voltage deviation. The Radau collocation method is employed to handle dynamics, by transforming all differential algebraic equations into algebraic ones. Most importantly, Pareto solutions are obtained through an accelerated multi-objective reinforcement learning (AMORL) method by filtering the dominated solutions. The entire feasible region is partitioned into small independent regions, to eliminate the scope for Pareto solutions. Besides, the AMORL method redefines the state functions and introduces creative state sensitivities, which accelerate the switch from learning to applying, once the agent accumulates sufficient knowledge. Furthermore, Pareto solutions are diversified via introducing some potential solutions. Lastly, the Fuzzy decision-making methodology picks up the tradeoff solution. Case studies are implemented on a practical 748-node power grid, which validate the acceleration and efficiency of the AMORL method. The AMORL method is overall superior to conventional reinforcement learning (RL) method with more optimal non-dominated objective values, much shorter CPU time, and better convergence to accurate values. Moreover, compared with another three state-of-the-art RL methods, the AMORL method takes almost the same CPU time of several seconds, but is slightly superior to the state-of-the-art methods in terms of optimal objective values. Additionally, the calculated values of the AMORL method fit the best with the accurate values during each iteration, resulting in a good convergence. INDEX TERMS Accelerated multi-objective reinforcement learning, dynamic optimization, Pareto solutions, short-term voltage security. NOMENCLATURE A. PARAMETERS

A practical guide to multi-objective reinforcement learning and planning

“…In addition to the motivating examples discussed above, recent years have seen multiobjective learning and planning methods applied across a wide range of problem domains including: distributed computing [27,124], drug and molecule design [62,214], cybersecurity [162], simulation [132], job shop scheduling [98], cognitive radio networks [100,129], satellite communications [45,63], recommender systems [78], power systems [34,35,97,193], building management [213], traffic management [70], manufacturing [36,54,80], bidding and pricing [76,207], education [151], and robotics [64]. The scope and variety of these applications supports our assertion that many important problems involve multiple objectives, and are best addressed using explicitly multi-objective methods.…”

A Practical Guide to Multi-Objective Reinforcement Learning and Planning

“…In addition to the motivating examples discussed above, recent years have seen multi-objective learning and planning methods applied across a wide range of problem domains including: distributed computing [Qin et al, 2020, da Silva Veith et al, 2019, drug and molecule design [Zhou et al, 2019, Horwood andNoutahi, 2020], cybersecurity [Sun et al, 2018], simulation [Ravichandran et al, 2018], job shop scheduling [Méndez-Hernández et al, 2019], cognitive radio networks [Messikh andZarour, 2018, Raj et al, 2020], satellite communications [Hu et al, 2020, Ferreira et al, 2019, recommender systems [Lacerda, 2017], power systems [Deng and Liu, 2018, Deng et al, 2020, Wang et al, 2019, Mello et al, 2020, building management , traffic management [Jin and Ma, 2019], manufacturing [Govindaiah and Petty, 2019, Lepenioti et al, 2020, Dornheim and Link, 2018, bidding and pricing [Yang et al, 2020, Krasheninnikova et al, 2019, education [Rowe et al, 2018], and robotics [Huang et al, 2019]. The scope and variety of these applications supports our assertion that many important problems involve multiple objectives, and are best addressed using explicitly multi-objective methods.…”

Modified tangent vector‐based voltage stability constrained optimal power flow considering limit‐induced bifurcation

“…The dynamic-based method includes a set of differential algebraic equations (DAEs) defining system dynamics [51][52][53][54]. It does not consider the required VSM but dynamic voltage stability.…”

“…Geng et al [52] proposed a hybrid method of dynamic simulation and direct method that solves discretised DAE as non‐linear programming. The authors in [53, 54] proposed a method converting DAE to non‐linear programming to be solved by artificial intelligence.…”