Rule pool updating through Sarsa‐learning to improve adaptability in changing environments

Abstract

Genetic network programming (GNP) is a new evolutionary algorithm using the directed graph as its chromosome. A GNPbased rule accumulation (GNP-RA) method was proposed previously for multiagent control. However, in changing environments where new situations appear frequently, the old rules in the rule pool become incompetent for guiding the agent's actions, and therefore updating them becomes necessary. This paper proposes a more robust rule-based model which can adapt to the environment changes. In order to realize this, Sarsa-learning is used as a tool to update the rules to cope with the unexperienced situations in new environments. Furthermore, Sarsa-learning helps to generate better rules by selecting really important judgments and actions during training. In addition, the ε-greedy policy of Sarsa enables GNP-RA to explore the solutions space sufficiently, generating more rules. Simulations on the tile world problem show that the proposed method outperforms the previous ones, namely GP and reinforcement learning.

Deep reinforcement learning for shared control of mobile robots

“…Sarsa‐learning is a great tool to update the policy in an unknown environment without any previous experience. It also helps generate better rules by selecting critical judgements and actions during training [14]. Nevertheless, in this approach, the continuous state and action spaces are generalized from the finite states and actions in previous experience [9].…”