Keyframe Sampling, Optimization, and Behavior Integration: Towards Long-Distance Kicking in the RoboCup 3D Simulation League

Abstract

Abstract. Even with improvements in machine learning enabling robots to quickly optimize and perfect their skills, developing a seed skill from which to begin an optimization remains a necessary challenge for large action spaces. This paper proposes a method for creating and using such a seed by i) observing the effects of the actions of another robot, ii) further optimizing the skill starting from this seed, and iii) embedding the optimized skill in a full behavior. Called KSOBI, this method is fully implemented and tested in the complex RoboCup 3D simulation domain. To the best of our knowledge, the resulting skill kicks the ball farther in this simulator than has been previously documented.

UT Austin Villa: RoboCup 2014 3D Simulation League Competition and Technical Challenge Champions

“…In winning the 2014 RoboCup competition UT Austin Villa finished with an undefeated record of 13 wins and 2 ties. 5 During the competition the team scored 52 goals without conceding any. Despite finishing with an undefeated record, the relatively few number of games played at the competition, coupled with the complex and stochastic environment of the RoboCup 3D simulator, make it difficult to determine UT Austin Villa being better than other teams by a statistically significant margin.…”

“…The 2014 UT Austin Villa agent includes sweeping changes to kicking from the 2013 agent. In addition to learning to kick further from a known starting point by mimicking another agent's existing kick as described in [5], the agent is also now able to reliably kick the ball after taking necessary steps to approach it. This latter improvement is achieved by learning a new kick approach walking parameter set for the team's omnidirectional walk engine, the purpose of which is to stop within a small bounding box of a target point while guaranteeing that the agent does not overshoot that target.…”

“…The robot is given 12 s to reach a position where it can kick the ball and is given the following reward during optimization: The 2014 team also adds the ability to legally score on indirect kickoffs. This is the result of a multiagent optimization where one agent uses a static, accurate, long-distance kick and the other attempts to touch the ball while moving it as little as possible [5].…”

“…At the RoboCup competition teams were given the option of using five different robot types with the requirement that at least three different types of robots must be used on a team, no more than seven of any one type, and no more than nine of any two types. The five types of robots available were the following: Table 3 shows performance metrics after optimizing both walks [6] and kicks [5] for all robot types. The general walk speed is for moving to different target positions on the field while the sprint speed is for walking forward to targets within 15 • of a robot's current heading.…”

Direct Loss Minimization Inverse Optimal Control

“…This paper discusses the implications of this framework for IOC and presents a series of experiments analyzing the performance of this methodology using a generic policy search framework based on the black box Covariance Matrix Adaptation (CMA) optimizer. CMA is quickly becoming a goto tool for complex nonlinear policy search problems [6] for its combined efficacy, simplicity, and strong theoretical connections to a very successful form of policy search known as PI 2 [26] which has been shown to perform well in realworld robotics applications [29].…”