Applications of reinforcement learning in energy systems

Perera, A.T.D.; Kamalaruban, Parameswaran

doi:10.1016/j.rser.2020.110618

Cited by 201 publications

(72 citation statements)

References 90 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike rule-based approaches, MPC approaches can have multiple objectives and constraints. MPC approaches are the most popular among DR and BEMS applications and have been used for many applications in the area of water heater and HVAC control [17]. For example, Tarragona [18] proposed an MPC strategy to improve the operation of a space-heating system coupled with renewable resources.…”

Section: Literature Reviewmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Water Heater Control

et al. 2021

View full text Add to dashboard Cite

Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Water Heater Control

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Amasyali and Mosavi showed how machine learning techniques can be applied for predicting energy consumption in various buildings [21,22]. Perera presented review studies in energy system applications by using reinforcement learning [23]. However, those review studies only concentrated on a single specific factor that influences the energy consumption of buildings (e.g., building control, electricity, and natural gas), or on a specific application (e.g., occupant behavior, load forecasting), or were restricted to the use of a specific intelligent computing technique (e.g., artificial neural networks and reinforcement learning), to classify and predict the energy consumption of buildings.…”

Section: Introductionmentioning

confidence: 99%

Machine Learning Techniques in the Energy Consumption of Buildings: A Systematic Literature Review Using Text Mining and Bibliometric Analysis

2021

View full text Add to dashboard Cite

The high level of energy consumption of buildings is significantly influencing occupant behavior changes towards improved energy efficiency. This paper introduces a systematic literature review with two objectives: to understand the more relevant factors affecting energy consumption of buildings and to find the best intelligent computing (IC) methods capable of classifying and predicting energy consumption of different types of buildings. Adopting the PRISMA method, the paper analyzed 822 manuscripts from 2013 to 2020 and focused on 106, based on title and abstract screening and on manuscripts with experiments. A text mining process and a bibliometric map tool (VOS viewer) were adopted to find the most used terms and their relationships, in the energy and IC domains. Our approach shows that the terms “consumption,” “residential,” and “electricity” are the more relevant terms in the energy domain, in terms of the ratio of important terms (TITs), whereas “cluster” is the more commonly used term in the IC domain. The paper also shows that there are strong relations between “Residential Energy Consumption” and “Electricity Consumption,” “Heating” and “Climate. Finally, we checked and analyzed 41 manuscripts in detail, summarized their major contributions, and identified several research gaps that provide hints for further research.

show abstract

“…The rule algorithm is direct and the heuristic method is simple, but in some cases, it is not the best solution [19]. Although the result of DDPG is less stable, but to deal with problems which are difficult to be completely controlled by model-based method with the increasing complexity of energy system, uncertainty, and security problems, it can work without specific model and quickly find solutions [20,21]. In addition, traditional industrial parks often have no ESS or only have little capacity photovoltaic panels.…”

Section: Introductionmentioning

confidence: 99%

Machine Learning Based Optimization Model for Energy Management of Energy Storage System for Large Industrial Park

Gao

Hong

2021

Processes

View full text Add to dashboard Cite

Renewable energy represented by wind energy and photovoltaic energy is used for energy structure adjustment to solve the energy and environmental problems. However, wind or photovoltaic power generation is unstable which caused by environmental impact. Energy storage is an important method to eliminate the instability, and lithium batteries are an increasingly mature technique. If the capacity is too large, it would cause waste and cost would increase, but too small capacity cannot schedule well. At the same time, the size of energy storage capacity is also constrained by power consumption, whereas large-scale industrial power consumption is random and non-periodic. This is a complex problem which needs a model that can not only dispatch but also give a reasonable storage capacity. This paper proposes a model considering the cycle life of a lithium battery and the installation parameters of the battery, and the electricity consumption data and photovoltaic power generation data of an industrial park was used to establish an energy management model. The energy management system aimed to reduce operating costs and obtain optimal energy storage capacity, which is constrained by lithium battery performance and grid demand. With the operational cost and reasonable battery capacity as the optimization objectives, the Deep Deterministic Policy Gradient (DDPG) method, the greedy dynamic programming algorithm, and the genetic algorithm (GA) were adopted, where the performance of lithium battery and the requirement of power grid were the constraints. The simulation results show that compared with the current forms of energy, the three energy management methods reduced the cost of capacity and operating of the energy storage system by 18.9%, 36.1%, and 35.9%, respectively.

show abstract

Applications of reinforcement learning in energy systems

Cited by 201 publications

References 90 publications

Deep Reinforcement Learning for Autonomous Water Heater Control

Deep Reinforcement Learning for Autonomous Water Heater Control

Machine Learning Techniques in the Energy Consumption of Buildings: A Systematic Literature Review Using Text Mining and Bibliometric Analysis

Machine Learning Based Optimization Model for Energy Management of Energy Storage System for Large Industrial Park

Contact Info

Product

Resources

About