A surrogate-assisted controller for expensive evolutionary reinforcement learning

Wang, Yuxing; Zhang, Tiantian; Chang, Yongzhe; Wang, Xueqian; Liang, Bin; Yuan, Bo

doi:10.1016/j.ins.2022.10.134

Cited by 9 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related to using surrogates in ERL, Wang et al in [24] proposed Surrogateassisted Controller (SC) applied on it to alleviate the computational burden of evaluation. Nevertheless, the core of SC is to replace the critic model, which aims to evaluate the action not the policy.…”

Section: Evolutionary Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

Tang¹,

Li²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditional reinforcement learning methods. However, the evaluation of the iteratively generated population usually requires a large amount of computational time and can be prohibitively expensive, which may potentially restrict the applicability of ERL. Surrogate is often used to reduce the computational burden of evaluation in EAs. Unfortunately, in ERL, each individual of policy usually represents millions of weights parameters of DNN. This high-dimensional representation of policy has introduced a great challenge to the application of surrogates into ERL to speed up training. This paper proposes a PE-SAERL Framework to at the first time enable surrogate-assisted evolutionary reinforcement learning via policy embedding (PE). Empirical results on 5 Atari games show that the proposed method can perform more efficiently than the four state-of-the-art algorithms. The training process is accelerated up to 7x on tested games, comparing to its counterpart without the surrogate and PE.

show abstract

Section: Evolutionary Reinforcement Learningmentioning

confidence: 99%

“…Therefore, the existing surrogate approaches may fail to accelerate the training of ERL. Few exceptions [6,20,24] employ the surrogates in the training phases, but they are not for high-dimensional DNN-based policies.…”

Section: Introductionmentioning

confidence: 99%