A Cooperative Charging Control Strategy for Electric Vehicles Based on Multiagent Deep Reinforcement Learning

Coordinated Electric Vehicle Active and Reactive Power Control for Active Distribution Networks

“…However, directly applying conventional DQN to each EV agent without extra information may suffer from instability issue, since all other agents' policies are implicitly formulated as part of the environment dynamics while their policies are continuously adjusted during the training process. To capture the other agents' control policy, a collective-policy model is proposed in [25] based on SAC that learns a fully decentralized policy for EVs charging strategy constrained by the transformer overload capacity. A hierarchical and hybrid MARL method based on proximal policy optimization (PPO) is proposed in [26] to optimize the multi-service provisions for EVs in a coupled power-transportation network.…”

A Multiagent Federated Reinforcement Learning Approach for Plug-In Electric Vehicle Fleet Charging Coordination in a Residential Community

“…For privacy concerns, agents may not be willing to share their local charging information with others. Consequently, a collective-policy model is proposed for each agent to approximate the collective behaviors [33]. Given the assumption that the function of collective power consumption is related to the changing electricity prices, the local approximation of the collective policy model can be described as,…”

“…Inspired by [33], the collective policy model is developed with a DNN and is expected to approximate the collected behaviors by training with historical electricity prices data and the collective power consumption received from the EVCS. Accordingly, each agent can acquire information regarding the collective actions of other agents and take them as a reference when making its own decision.…”

“…When overload occurs, each household agent will receive a penalty depending on its energy consumption during the overload hours. It can be treated as the cost for coordination described by [33]…”

“…FIO (Full knowledge observable algorithms): The benchmark method of FIO is commonly introduced for comparison in DRL-based optimization problems, primarily following [31] and [33], in which this approach is adopted solve a similar electric vehicle charging strategy optimization problem. Specifically, the benchmarked algorithm is based on an accurate PEVs scheduling model assuming perfect observation of the environment.…”