Probabilistic Guided Exploration for Reinforcement Learning in Self-Organizing Neural Networks

Abstract

Exploration is essential in reinforcement learning, which expands the search space of potential solutions to a given problem for performance evaluations. Specifically, carefully designed exploration strategy may help the agent learn faster by taking the advantage of what it has learned previously. However, many reinforcement learning mechanisms still adopt simple exploration strategies, which select actions in a pure random manner among all the feasible actions. In this paper, we propose novel mechanisms to improve the existing knowledgebased exploration strategy based on a probabilistic guided approach to select actions. We conduct extensive experiments in a Minefield navigation simulator and the results show that our proposed probabilistic guided exploration approach significantly improves the convergence rate.

Exploration in Deep Reinforcement Learning: A Survey

“…Another approach of this type could be reducing states for exploration based on some predefined metric. An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152]. In ART, knowledge about actions can be split into: (i) positive chunk which leads to positive rewards, (ii) negative chunk which leads to negative results, and (iii) empty chunk which is not yet taken.…”

“…In this approach, the action is randomly chosen from positive and no chunks; thus, the agent is exploring either new things or ones with the positive reward. Wang et al [152] extended this to include the probability of selecting the remaining actions based on how well they are known.…”