Trajectory Tracking With an Aggregation of Domestic Hot Water Heaters: Combining Model-Based and Model-Free Control in a Commercial Deployment

Abstract

Scalable demand response of residential electric loads has been a timely research topic in recent years. The commercial coming of age or residential demand response requires a scalable control architecture that is both efficient and practical to use. This work presents such a strategy for domestic hot water heaters and present a commercial proof-of-concept deployment. The strategy combines state of the art in aggregate-and-dispatch with a novel dispatch strategy leveraging recent developments in reinforcement learning and is tested in a hardware-in-theloop simulation environment. The results are promising and present how model-based and model-free control strategies can be merged to obtain a mature and commercially viable control strategy for residential demand response.Index Terms-Thermostatically controlled load, model predictive control, demand response, reinforcement learning.

Domain Randomization for Demand Response of an Electric Water Heater

“…All symbols are defined in Table I and II. Since the water content of the buffer has a uniform temperature SoC is defined as in (2).…”

“…Due to its proven track record in DR applications we also consider FQI [1,2,16] in our experiments. In a discrete-time MDP with a one day horizon (T = 96) we can reformulate the SDP as a sequence of T control problems.…”

“…In the past, residential Thermostatically Controlled Loads (TCLs) were identified as promising appliances for DR, mainly due to their inherent flexibility emerging from their decoupled electricity and heat demand [1]. But, a wide variety of TCLs exist, thus amplifying the need for scalable control [2].…”

“…During their four month real-life demonstration with six residential buildings they observed an increase of 20% in total PV self-consumption. In a third application, Liu et al [2] use an aggregation of EWHs to balance wind power generation. Our fourth and final example of DR applications consists of the work of Kazmi et al [6], in which the authors employ RL to increase energy efficiency of EWHs.…”

Determinants of energy flexibility in residential hot water systems

“…The energy consumed for hot water production accounts for between 10% and 25% of end energy demand in many countries [10], [11] and will gain in relative importance because of improvements in building envelopes reducing the share of space conditioning [11], [12]. Operational control of hot water systems has been explored in existing literature extensively, both in the context of optimizing local objectives such as energy efficiency [13], [14] and global objectives such as grid supportive behaviour [15], [16], [17]. Some examples of grid supportive behaviour include peak shaving, valley filling and maximizing self-consumption of local renewable energy.…”