Ensemble Reinforcement Learning: A Survey

Abstract

Reinforcement Learning (RL) has emerged as a highly effective technique for addressing various scientific and applied problems. Despite its success, certain complex tasks remain challenging to be addressed solely with a single model and algorithm. In response, ensemble reinforcement learning (ERL), a promising approach that combines the benefits of both RL and ensemble learning (EL), has gained widespread popularity. ERL leverages multiple models or training algorithms to comprehensively explore the problem space and possesses strong generalization capabilities. In this study, we present a comprehensive survey on ERL to provide readers with an overview of recent advances and challenges in the field. First, we introduce the background and motivation for ERL. Second, we analyze in detail the strategies that have been successfully applied in ERL, including model averaging, model selection, and model combination. Subsequently, we summarize the datasets and analyze algorithms used in relevant studies. Finally, we outline several open questions and discuss future research directions of ERL. By providing a guide for future scientific research and engineering applications, this survey contributes to the advancement of ERL.

Structure in Deep Reinforcement Learning: A Survey and Open Problems

“…Compositional Policies Continual settings utilize policies compositionally by treating already learned policies as primitives. Such methods feed these primitives to the discrete optimization problems for selection mechanisms or to continuous optimization settings involving ensembling (Song et al, 2023) and distillation (Rusu et al, 2016). Modularity in such settings manifests itself by construction and is a central factor in building solutions.…”