Analysis and Adaptation of Q-Learning Algorithm to Expert Controls of a Solar Domestic Hot Water System

Abstract

This paper discusses the development of a coupled Q-learning/fuzzy control algorithm to be applied to the control of solar domestic hot water systems. The controller brings the benefit of showing performance in line with the best reference controllers without the need for devoting time to modelling and simulations to tune its parameters before deployment. The performance of the proposed control algorithm was analysed in detail concerning the input membership function defining the fuzzy controller. The algorithm was compared to four standard reference control cases using three performance figures: the seasonal performance factor of the solar collectors, the seasonal performance factor of the system and the number of on/off cycles of the primary circulator. The work shows that the reinforced learning controller can find the best performing fuzzy controller within a family of controllers. It also shows how to increase the speed of the learning process by loading the controller with partial pre-existing information. The new controller performed significantly better than the best reference case with regard to the collectors’ performance factor (between 15% and 115%), and at the same time, to the number of on/off cycles of the primary circulator (1.2 per day down from 30 per day). Regarding the domestic hot water performance factor, the new controller performed about 11% worse than the best reference controller but greatly improved its on/off cycle figure (425 from 11,046). The decrease in performance was due to the choice of reward function, which was not selected for that purpose and it was blind to some of the factors influencing the system performance factor.

Comparison of Solar Collector Testing Methods—Theory and Practice

“…The proper determination of the static and dynamic properties of a solar collector is of key significance, as they constitute a basis for the design of a solar heating installation, as well as a control system. In spite of the continuing development of the concepts of adjusting the operation of a solar heating installation based on the simplest proportional controller [9], proportional and on-off fluid flow control algorithm [10], proportional-derivative control algorithm [11], through continuous adjustment using a Proportional Integral Differential PID regulator [12,13] and ending with complicated fuzzy algorithms [14,15], learning algorithm [16], and artificial neural networks [17] improving the performance of thermal solar collector, the ensuring of required control quality still remains an issue. They probably arise from the fact that the static and dynamic parameters of solar collectors determined in laboratory tests can be different under operating conditions.…”