Accelerating Q-Learning through Kalman Filter Estimations Applied in a RoboCup SSL Simulation

Deep-Q-Network Hybridization with Extended Kalman Filter for Accelerate Learning in Autonomous Navigation with the Auxiliary Security Module

“…The authors in 12 present a study on the acceleration of reinforcement learning using KF and were pioneers in this type of application. The authors' objective was to train a goalkeeper for robot soccer, in the championship known in the community as RoboCup.…”

“…According to 12 , one of the problems of reinforcement learning is the speed of convergence. Once the agent learns by iterations, if the exploration space, that is, the space in which the agent interacts with the environment is highly dimensional (there are several possible states), it is necessary to increase the number of training episodes and consequently make it more expensive.…”

Batch Reinforcement Learning on a RoboCup SSL Keepaway Strategy Learning Problem

“…Although we tackle the problem of finding a keepaway strategy, several challenges arises at the Small Size League in addition to the already mentioned problems like goalkeeper training on (Ahumada et al, 2013), learning the opponent strategy as on (Yasui et al, 2013),or learning to control the dribbler (Riedmiller et al, 2008), noting that the work therein focuses on the Middle Size League. This latter problem also applies to the Small Size League, being specially difficult to…”

“…In this problem, a team of cooperative agents have to play a soccer match against another team composed of autonomous agents, noting that a possible objective for a given team could be to keep as far as possible the ball from its own goal area. In order to achieve this objective, many works can be found in the literature, from keepaway strategies using a multi-agent approach (Stone et al, 2005) to algorithms focused on training just the goalkeeper, as (Ahumada et al, 2013) or (Celiberto et al, 2007). This work, like in (Ahumada et al, 2013) and (Celiberto et al, 2007) uses a grid for discretizing the state space of the agent and therefore avoiding to deal with a continous state space representation where tabular methods become impractical (Baird and Klopf, 1993).…”

AIS: Artificial Intelligent Soccer

“…This 18 RoboCup team belongs to one of several projects from this aggroupation and is conformed by 19 students with different specialization areas such as computer science, control, and automation, or 20 power electronics, but also on a multidisciplinary approach including students from mechanical 21 engineering, as well as industrial engineering students. 22 This SSL team follows the nature of the host students initiative, starting from its multidisci-23 plinary constitution, the self-organization and motivation with professor advises when required but 24 managed independently from any professor funding project, and trascendence over generations 25 renewing its members with a constantly growing development and enhancement, and making 26 both research and development works, like [1] where a previous generation of the team applied 27 reinforcement learning on the goalkeeper task. 28 This document describes our design and the implementation we have got so far, showing all the 29 work made in the different areas involved in this category.…”