In this paper, we present a scalable approach for DSM (demand side management) of PHEVs (plug-in hybrid electric vehicles). Essentially, our approach consists of three steps: aggregation, optimization, and control. In the aggregation step, individual PHEV charging constraints are aggregated upwards in a tree structure. In the optimization step, the aggregated constraints are used for scalable computation of a collective charging plan, which minimizes costs for electricity supply. In the real-time control step, this charging plan is used to create an incentive signal for all PHEVs, determined by a market-based priority scheme. These three steps are executed iteratively to cope with uncertainty and dynamism. In simulation experiments, the proposed three-step approach is benchmarked against classic, fully centralized approaches. Results show that our approach is able to charge PHEVs with comparable quality to optimal, centrally computed charging plans, while significantly improving scalability. Index Terms-Demand side management, market-based control, plug-in hybrid electric vehicles.
Driven by recent advances in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL to demand response. In contrast to conventional modelbased approaches, batch RL techniques do not require a system identification step, making them more suitable for a large-scale implementation. This paper extends fitted Q-iteration, a standard batch RL technique, to the situation when a forecast of the exogenous data is provided. In general, batch RL techniques do not rely on expert knowledge about the system dynamics or the solution. However, if some expert knowledge is provided, it can be incorporated by using the proposed policy adjustment method. Finally, we tackle the challenge of finding an open-loop schedule required to participate in the day-ahead market. We propose a model-free Monte Carlo method that uses a metric based on the state-action value function or Q-function and we illustrate this method by finding the day-ahead schedule of a heat-pump thermostat. Our experiments show that batch RL techniques provide a valuable alternative to model-based controllers and that they can be used to construct both closed-loop and open-loop policies.
Abstract-This paper addresses the problem of defining a day-ahead consumption plan for charging a fleet of electric vehicles (EVs), and following this plan during operation. A challenge herein is the beforehand unknown charging flexibility of EVs, which depends on numerous details about each EV (e.g., plug-in times, power limitations, battery size, power curve, etc.). To cope with this challenge, EV charging is controlled during opertion by a heuristic scheme, and the resulting charging behavior of the EV fleet is learned by using batch mode reinforcement learning. Based on this learned behavior, a cost-effective day-ahead consumption plan can be defined. In simulation experiments, our approach is benchmarked against a multistage stochastic programming solution, which uses an exact model of each EVs charging flexibility. Results show that our approach is able to find a day-ahead consumption plan with comparable quality to the benchmark solution, without requiring an exact day-ahead model of each EVs charging flexibility.Index Terms-Demand-side management, electric vehicles (EVs), reinforcement learning (RL), stochastic programming (SP). NOMENCLATUREThe symbols and notations used throughout this paper are summarized below. Manuscript received October 15, 2013; revised March 20, 2014 and August 28, 2014; accepted October 20, 2014 Offset from day-ahead prices, to define imbalance prices. Functions T(h)Mapping from market period h to the set of control periods in market period h. f heur Heuristic function to dispatch power to EVs. Real variables E da hEnergy bought in the day-ahead market for market period h. Energy charged by the EV fleet in control period t. i P n t Charging power of EV i during control period t in senario n.
This paper addresses the challenges of integrating existing PHEV charging algorithms, which optimize PHEV charging per market timeslot (e.g. 15 minutes), into an environment with realistic communication conditions. To address this challenge, we propose a dual coordination mechanism, which controls a cluster of devices on two different operation levels: market operation and real-time operation. The market operation level uses an existing timeslot-based algorithm to calculate a charging schedule per timeslot. The real-time operation level translates this schedule into event-based control actions for a realistic communication environment, wherein a limited number of messages can be exchanged. A case study of 1000 PHEVs shows that it is possible to achieve results on par with the timeslot based algorithm but with significantly reduced communication with the PHEVs.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.