Optimal management of energy storage system based on reinforcement learning

Liu, Jing; Tang, Hao; Matsui, Masayuki; Takanokura, Masato; Zhou, Lihua; Gao, Xueying

doi:10.1109/chicc.2014.6896376

Cited by 4 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the "SARSA" algorithm, a control operation was developed to maximize long-term rewards to encounter the load demand. The results in [10] showed a significant increase of total rewards, a faster convergence speed and good effect. A reinforcement learning controller developed and simulated using the MATLAB/Simulink environment shows that as compared to the other controllers, the reinforcement learning controller has equivalent or better performance even after a couple of simulated years [11].…”

Section: Literature Reviewmentioning

confidence: 88%

Optimal Control Policy for Energy Management of a Commercial Bank

2022

View full text Add to dashboard Cite

There has been substantial research on Building Energy Management Systems. Most of the work has focused more on the management scheme and less on the specific structure or the nature of activities within each building. However, recently some attention is being paid to these specifics, and this paper is one of such efforts, where we consider the structure and nature of activities in the building for developing an energy management system custom designed for a bank branch where customers may arrive randomly based on a known probability distribution. Specifically, this paper presents a model for generating an optimal control policy to manage the electrical energy of a commercial bank building. A Markov Decision Process (MDP) model is proposed. The MDP model is solved for the calculation of an optimal control policy using stochastic dynamic programming. The advantage of the proposed model is that it can incorporate uncertainty involved in the problem. Another advantage is that the output control policy is optimal with respect to a discounted cost/reward function. A disadvantage of the proposed scheme is computational complexity. To overcome this disadvantage, a decomposition-based approach is proposed. A unique feature of the proposed MDP-based model is that it was developed for a specific type of building, i.e., a bank. The need for a Building Management System (BMS) that is specific for a particular type of building arises due to the fact that each building has its own working parameters and environment. Our focus is to give a customized BMS framework for a bank building. Practical implementation of the developed model is discussed and a case study is included for demonstration purposes. Results obtained from the case study indicate that considerable savings in the electrical energy expenditure can be achieved without compromising comfort. This is possible due to optimization of the control policy using the statistical information relevant to the problem.

show abstract

Section: Literature Reviewmentioning

confidence: 88%

Optimal Control Policy for Energy Management of a Commercial Bank

2022

View full text Add to dashboard Cite

show abstract

“…To solve (15), we can adopt certain RL algorithms to search for the optimal parameter θ; see e.g., [13]. We use the deep Qnetworks (DQNs) [20,Ch.…”

Section: Optimal Battery Control Algorithmmentioning

confidence: 99%

“…This data-driven framework helps to bypass the hurdles in formulating the complex models of system dynamics or estimating the statistical information on the uncertainty. Specifically for the OBC problem, it allows to flexibly incorporate a variety of operational objectives, and several RL techniques have been widely used, such as the Deep Q-Network (DQN) [14], SARSA [15] and T D(λ)-learning [16]. However, a majority of these techniques have not considered the battery degradation cost, as its cycle-based model is difficult to include by the RL formulation.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning Based Optimal Battery Control Under Cycle-based Degradation Cost

Kwon¹,

Zhu²

2021

Preprint

View full text Add to dashboard Cite

Battery energy storage systems are providing increasing level of benefits to power grid operations by decreasing the resource uncertainty and supporting frequency regulation. Thus, it is crucial to obtain the optimal policy for battery to efficiently provide these grid-services while accounting for its degradation cost. To solve the optimal battery control (OBC) problem using the powerful reinforcement learning (RL) algorithms, this paper aims to develop a new representation of the cycle-based battery degradation model according to the rainflow algorithm. As the latter depends on the full trajectory, existing work has to rely on linearized approximation for converting it into instantaneous terms for the Markov Decision Process (MDP) based formulation. We propose a new MDP form by introducing additional state variables that can easily keep track of past switching points for determining the cycle depth. The proposed degradation model allows to adopt the powerful deep Q-Network (DQN) based RL algorithm to efficiently search for the OBC policy. Numerical tests using real market data have demonstrated the performance improvements of the proposed cycle-based degradation model in enhancing the battery operations while mitigating its degradation, as compared to earlier work using the linearized approximation.

show abstract

Research on Demand Response Strategy of HVAC Based on Deep Reinforcement Learning

Yan

Cui

et al. 2020

2020 5th International Conference on Power and Renewable Energy (ICPRE)

View full text Add to dashboard Cite

Optimal management of energy storage system based on reinforcement learning

Cited by 4 publications

References 10 publications

Optimal Control Policy for Energy Management of a Commercial Bank

Optimal Control Policy for Energy Management of a Commercial Bank

Reinforcement Learning Based Optimal Battery Control Under Cycle-based Degradation Cost

Research on Demand Response Strategy of HVAC Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About