Neural reinforcement learning for an obstacle avoidance behavior

Abstract

Reinforcement learning (RL) offers a set of various algorithms for in-situation behavior synthesis [1]. The Qlearning [2] technique is certainly the most used of the RL methods. Multilayer perceptron implementations of the Q-learning have been proposed early [3], due to the interest of the restricted memory need and the generalization capability [4]. Self-organizing map implementation of the Q-learning is more recent [5]. We propose to study the use and discuss the interest of this implementation comparing to a multilayer perceptron implementation or more classical ones. Experiments are performed in the real world with the miniature robot Khepera [6]. Q-learningReinforcement learning synthesises a mapping function between situations and actions by minimising a reinforcement signal. Q-learning algorithms store the expected reinforcement value associated to each situationaction pair. Three different functions are involved: memorisation, exploration and updating [4]. In respond to the present situation, an action is proposed by the robot memory. This action is the one that has the best rewarding probability. However, this proposition is eventually modified to allow an extensive exploration of the situationaction space. After the execution by the robot of the action in the real world, a reinforcement function provides a reinforcement value. This value, a simple qualitative criterion (+1, -1 or 0), is used by the updating algorithm to adjust the reward value (Q) associated to the situation-action pair. The learning is incremental, because the acquisition of the examples is carried out in real situations.

Reinforcement learning-based group navigation approach for multiple autonomous robotic systems

“…However, when dealing with a real robot the state space is typically too large to explore. A solution to this problem is to introduce decision trees or neural networks [10,34,35] to approximate the value function (state-action function) in order to generalize from a certain number of situations.…”

“…When the situation space becomes so large combined with all possible actions, an exhaustive exploration or memorization of all situation-action pairs is impossible. One solution to this problem is the generalization process through the use of artificial neural networks, as suggested in Refs [29,34,35], but problems with high-dimensional movement systems remain daunting [39]. A possible way to reduce the computational complexity of learning a control policy comes from modularizing the policy.…”

A disembodied developmental robotic agent called Samu Bátfai

“…Sigmoidal multilayer perceptron (MLP) neural networks are used to approximate the Q-function. In accordance with the QCON model [22], [30] Figure 2: This is a simplified code snippet that is iterated over all linkages of the parsed sentence returned by the Link Grammar [1]. The 'words' is a vector of strings that contains all words of a given linkage.…”