Multi-Agent Cooperation Based Reduced-Dimension Q(λ) Learning for Optimal Carbon-Energy Combined-Flow

Abstract

This paper builds an optimal carbon-energy combined-flow (OCECF) model to optimize the carbon emission and energy losses of power grids simultaneously. A novel multi-agent cooperative reduced-dimension Q(λ) (MCR-Q(λ)) is proposed for solving the model. Firstly, on the basis of the traditional single-objective Q(λ) algorithm, the solution space is reduced effectively to shrink the size of Q-value matrices. Then, based on the concept of ant cooperative cooperation, multi-agents are used to update the Q-value matrices iteratively, which can significantly improve the updating rate. The simulation in the IEEE 118-bus system indicates that the proposed technique can decrease the convergence speed by hundreds of times as compared with conventional Q(λ), keeping high global stability, which is very suitable for dynamic OCECF in a large and complex power grid compared with other algorithms.

Deep reinforcement learning based power system optimal carbon emission flow

“…Furthermore, based on the optimal power flow (OPF) (Momoh et al, 1999a;Momoh et al, 1999b) and combined with the CEF analysis theory, researchers have proposed the optimal carbon emission flow (OCEF) model of the power system (Zhang et al, 2015;Cao et al, 2020), which takes into account the minimization of power generation cost and CEFL under the condition that the system's safety constraints are met. The OCEF problem is a complex nonconvex nonlinear programming problem, similar to the OPF problem.…”