On-Line Building Energy Optimization Using Deep Reinforcement Learning

Abstract

Unprecedented high volumes of data are becoming available with the growth of the advanced metering infrastructure. These are expected to benefit planning and operation of the future power system, and to help the customers transition from a passive to an active role. In this paper, we explore for the first time in the smart grid context the benefits of using Deep Reinforcement Learning, a hybrid type of methods that combines Reinforcement Learning with Deep Learning, to perform on-line optimization of schedules for building energy management systems. The learning procedure was explored using two methods, Deep Q-learning and Deep Policy Gradient, both of them being extended to perform multiple actions simultaneously. The proposed approach was validated on the large-scale Pecan Street Inc. database. This highly-dimensional database includes information about photovoltaic power generation, electric vehicles as well as buildings appliances. Moreover, these on-line energy scheduling strategies could be used to provide realtime feedback to consumers to encourage more efficient use of electricity.

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

“…An energy optimization problem in a smart grid is formulated in [100]. An on-line energy scheduling strategy is learned using deep Q-learning and deep policy gradient methods.…”

A Learning-Based Power Management Method for Networked Microgrids Under Incomplete Information

“…Here, λ F t,n is the diesel generator fuel price in $/L adopted from [32]. The fuel consumption F i,t,n of diesel generator can be expressed as a quadratic polynomial function (17), with coefficients a f = 0.0001773 L/kW 2 , b f = 0.1709 L/kW , and c f = 14.67L adopted from [33]. Constraints (18) .…”

Multi-agent Deep Reinforcement Learning for Zero Energy Communities

“…The general problem with majority of the algorithms is that, for optimization they compute partial or the entire solution space to choose the best one, and hence are time consuming. [5] explores an interesting approach that avoids computing the entire search space using Deep Reinforcement Learning (DRL) to optimize schedules for building's energy management systems. The authors investigate two DRL based algorithms, Deep Policy Gradient (DPG) and Deep Q-Learning (DQN) for building a on-line large scale solution and conclude it to be more effective than traditional solutions.…”

“…We model this community as a multi-agent environment where each agent represents a building. Previous works [5] have demonstrated that deep reinforcement learning (DRL) is an effective technique for energy management in a single building management system. Considering this, we propose a DRL-based solution to optimize energy sharing between multiple such buildings.…”