Optimal control in microgrid using multi-agent reinforcement learning

Li, Fudong; Wu, Min; He, Yong; Chen, Xin

doi:10.1016/j.isatra.2012.06.010

Cited by 70 publications

(34 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Q*(s,a) is defined as follows [13]: Where s is current state; s ' is next state; is discount factor; Pss ' (s ' |s,a) is the probability to reach state s' if action a is done in state s; r(s,s ' ,a) is reward value the agent will get. Pss ' (s ' |s,a) and r(s,s ' ,a) is uncertainty since the selected action is unknown the next moment, so each Q(s,a) approximates Q*(s,a) by iteration.…”

Section: Q-learing Algorithmmentioning

confidence: 99%

“…Under grid-connected mode, dynamic hierarchical reinforcement learning is established to minimize electricity costs on the premise of satisfying generation limits of units and power balance among power production and consumption in a microgrid [12]. A two steps-ahead reinforcement learning algorithm is proposed to make use of time-dependent environmental experience and optimize the battery scheduling in an energy management system for a microgrid [13].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Q-learning algorithm based multi-agent coordinated control method for microgrids

Yuanyuan

Chang

Mao

et al. 2015

2015 9th International Conference on Power Electronics and ECCE Asia (ICPE-ECCE Asia)

View full text Add to dashboard Cite

This paper proposes a Q-learning algorithm (Q-LA) based multi-agent coordinated control method for microgrids. By the method, Q-LA is adopted to calculate the power to be regulated, which is called the microgrid regulation error (MRE), in secondary control for real-time operation. And the generation schedule of distributed generators (DGs) as well as batteries is modified in real time with the MRE by the fuzzy theory and particle swarm optimization method, taking the economy and environmental benefits into consideration together. The simulation platform of Q-LA based multi-agent hybrid energy management system for microgrid (HEMS-MG) is established in C++ Builder. The simulation results verify the effectiveness and feasibility of the proposed method.

show abstract

Section: Q-learing Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Q-learning algorithm based multi-agent coordinated control method for microgrids

Yuanyuan

Chang

Mao

et al. 2015

2015 9th International Conference on Power Electronics and ECCE Asia (ICPE-ECCE Asia)

View full text Add to dashboard Cite

show abstract

“…In contrast to the conventional distributed methods, learning-based methods can be easily adapted with a real-time problem after the off-line training process. In RL, Q-learning is a popular method and is widely used for the optimal operation of microgrids [19][20][21][22][23]. A fitted Q-iteration-based algorithm has been proposed in [19] for a BESS.…”

Section: Introductionmentioning

confidence: 99%

“…By using this method, the utilization rate of the battery is increased during high electricity demand while the utilization rate of the wind turbine for local demand is also increased to reduce the consumer dependence on the utility grid. The authors in [23] have presented an improved RL method to minimize the operation cost of an MG in the grid-connected mode.…”

Section: Introductionmentioning

confidence: 99%

“…Adjacent MGs can be interconnected to form a multi-microgrid system to improve network reliability by sharing power among MGs and other community entities [11]. However, the power transfers between other community entities and among MGs of the network have not been considered in the existing Q-learning-based operation methods [19][20][21][22][23]. Therefore, the existing methods are not suitable to apply for multi-microgrid systems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Q-Learning-Based Operation Strategy for Community Battery Energy Storage System (CBESS) in Microgrid System

2019

View full text Add to dashboard Cite

Energy management systems (EMSs) of microgrids (MGs) can be broadly categorized as centralized or decentralized EMSs. The centralized approach may not be suitable for a system having several entities that have their own operation objectives. On the other hand, the use of the decentralized approach leads to an increase in the operation cost due to local optimization. In this paper, both centralized and decentralized approaches are combined for managing the operation of a distributed system, which is comprised of an MG and a community battery storage system (CBESS). The MG is formed by grouping all entities having the same operation objective and is operated under a centralized controller, i.e., a microgrid EMS (MG-EMS). The CBESS is operated by using its local controller with different operation objectives. A Q-learning-based operation strategy is proposed for optimal operation of CBESS in both grid-connected and islanded modes. The objective of CBESS in the grid-connected mode is to maximize its profit while the objective of CBESS in islanded mode is to minimize the load shedding amount in the entire system by cooperating with the MG. A comparison between the Q-learning-based strategy and a conventional centralized-based strategy is presented to show the effectiveness of the proposed strategy. In addition, an adjusted epsilon is also introduced for epsilon-greedy policy to reduce the learning time and improve the operation results.

show abstract

Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Sanusi

Mills

Dodd

et al. 2020

Adaptive Control & Signal

View full text Add to dashboard Cite

SummaryConventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach.

show abstract

Optimal control in microgrid using multi-agent reinforcement learning

Cited by 70 publications

References 19 publications

Q-learning algorithm based multi-agent coordinated control method for microgrids

Q-learning algorithm based multi-agent coordinated control method for microgrids

Q-Learning-Based Operation Strategy for Community Battery Energy Storage System (CBESS) in Microgrid System

Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Contact Info

Product

Resources

About