A reinforcement learning approach for optimal heating curve adaption

Cascaded reinforcement learning based supply temperature control

“…To obtain learning data for the reinforcement learning based control design, a simplified Modelica model of the building's upper floor and its heating network is developed according to [10]. The modelling tool SimulationX and the GreenCity Library [12] is used.…”

“…The following system variables can be defined as the states for the supply temperature control [10]: Current ambient temperature 𝑠𝑠 𝑇𝑇 𝑎𝑎 , current zone temperature 𝑠𝑠 𝑇𝑇 𝑧𝑧 , current hour of day 𝑠𝑠 𝐻𝐻𝐻𝐻𝐻𝐻 , current week day 𝑠𝑠 𝑊𝑊 , and predicted solar energy of an upcoming time horizon 𝑠𝑠 𝐹𝐹𝐹𝐹𝐻𝐻𝐹𝐹 , as well as predicted mean ambient temperature upcoming time horizon 𝑠𝑠 𝐹𝐹𝐹𝐹𝑇𝑇𝐹𝐹 . They give rise to a state vector 𝒔𝒔.…”

“…Eventually, a reward function needs to be defined. As the goal for the demonstrator building is to maximize comfort, the deviation of zone temperature 𝑇𝑇 𝑧𝑧𝐻𝐻𝑧𝑧𝑧𝑧 from its setpoint 𝑇𝑇 𝑧𝑧𝐻𝐻𝑧𝑧𝑧𝑧,𝑠𝑠 is computed [10],…”

“…In this contribution, a reinforcement learning based supply temperature control approach -Q-learning (QL) -is proposed. Based on the work in [10] the controller is designed such that occupation and weather prediction are taken into account. Further, in order to deal with thermal effects of different time constants, the control design is extended with a cascaded structure.…”

“…Its modelling is presented which is used to generate learning data for the control design. In Section 3 the reinforcement learning control from [10] is briefly described and simulation results are discussed. In Section 4 a cascaded control design is introduced and evaluated followed by simulation results.…”