Jointly Learning Environments and Control Policies with Projected Stochastic Gradient Ascent

Abstract

In this work, we generalize the direct policy search algorithms to an algorithm we call Direct Environment Search with (projected stochastic) Gradient Ascent (DESGA). The latter can be used to jointly learn a reinforcement learning (RL) environment and a policy with maximal expected return over a joint hypothesis space of environments and policies. We illustrate the performance of DESGA on two benchmarks. First, we consider a parametrized space of Mass-Spring-Damper (MSD) environments. Then, we use our algorithm for optimizing the size of the components and the operation of a small-scale and autonomous energy system, i.e. a solar off-grid microgrid, composed of photovoltaic panels, batteries, etc. The results highlight the excellent performances of the DESGA algorithm.Preprint. Under review.

Usage of GAMS-based Digital Twins and Clustering to Improve Energetic Systems Control

“…Several approaches are being developed for energy management systems [6][7][8][9]27,28]. The latest one, based on deep reinforced learning, while producing results and being developed in open-sourced frameworks [29][30][31][32] is considered a black-box model [33] and the results are hard to explain, analyze and validate for domain experts. The approach developed in this paper can be considered as a proxy for Off-Policy reinforcement learning [34].…”