Instance-Based Action Models for Fast Action Planning

Abstract

Two main challenges of robot action planning in real domains are uncertain action effects and dynamic environments. In this paper, an instance-based action model is learned empirically by robots trying actions in the environment. Modeling the action planning problem as a Markov decision process, the action model is used to build the transition function. In static environments, standard value iteration techniques are used for computing the optimal policy. In dynamic environments, an algorithm is proposed for fast replanning, which updates a subset of state-action values computed for the static environment. As a test-bed, the goal scoring task in the RoboCup 4-legged scenario is used. The algorithms are validated in the problem of planning kicks for scoring goals in the presence of opponent robots. The experimental results both in simulation and on real robots show that the instance-based action model boosts performance over using parametric models as done previously, and also incremental replanning significantly improves over original off-line planning.

A Markov Framework for Learning and Reasoning About Strategies in Professional Soccer

“…To train the humanoids, they specify multiple training tasks corresponding to soccer drills, which they model as stochastic games. Additionally, Ahmadi and Stone (2008) and Bai et al (2012) investigate different automated action planning strategies for in-game decision-making. However, such automated planning strategies cannot immediately be used in real-life soccer as it is compounded by high stakes and the incredibly short tenure of most managers, which discourages experimentation.…”

Learning Powerful Kicks on the Aibo ERS-7: The Quest for a Striker

“…They subsequently use this knowledge to select the most appropriate kick for a given situation, effectively reducing the time taken for an Aibo to score a goal. Similarly, Ahmadi and Stone [2] investigate kick models for use in the action planning domain. Unlike the parameterized kick models used by Chernova and Veloso, Ahmadi employs instance based models, in which the resulting ball location of each kick is recorded.…”