Learning to Run Faster in a Humanoid Robot Soccer Environment Through Reinforcement Learning

Abreu, Miguel; Reis, Luís Paulo; Lau, Nuno

doi:10.1007/978-3-030-35699-6_1

Cited by 33 publications

(32 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is possible since in a kick scenario, moving the head joints brings little Early attempts tried to directly relate the values of the action space to the angular velocities, using variations on a simple proportional or a complete PID controller. Nevertheless, as also verified in previous work [12], a noticeable improvement is obtained through interpreting each term of the action space as relating to an angle, and from there deriving an angular velocity to feed as an input. In the end, the desired objective angle θ goal , can be obtained from the action space output as: With θ goal now defined we can now obtain from it the angular velocity as:…”

Section: Action Spacesupporting

confidence: 73%

“…This forward motion corresponds to one of several skills developed for usage in a game environment. These correspond to a sprinting and running pace [12].…”

Section: Motivationmentioning

confidence: 99%

“…This component is comprised of the skills that generate the motion from which the kick follows. These skills were previously developed by Miguel Abreu using a PPO algorithm [12].…”

Section: Skillsmentioning

confidence: 99%

“…Previous work done by the FCPortugal3D team has successfully sidestepped this issue by only considering every third tick for the simulation, and performing the same action for those where data is not up to date [12,83]. Unfortunately, given the nature of the kick, and the short timeframes it entails, this sort of granularity was not acceptable.…”

Section: Observation Spacementioning

confidence: 99%

See 3 more Smart Citations

Humanoid Robot Kick in Motion Ability for Playing Robotic Soccer

Teixeira

Silva

Abreu

et al. 2020

2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC)

Self Cite

View full text Add to dashboard Cite

Robotics and Artificial Intelligence are two deeply intertwined fields of study, currently experiencing formidable growth. To foster these developments, the RoboCup initiative is a fantastic test bed to experiment new approaches and ideas. This dissertation is rooted in the groundwork laid by previous FCPortugal3D contributions for the RoboCup simulation 3D robotic soccer league, and seeks to design and implement a humanoid robotic kick system for situations where the robot is moving. It employs Reinforcement Learning (RL) techniques, namely the Proximal Policy Optimization (PPO) algorithm to create fast and reliable skills. The kick was divided into 6 cases according to initial conditions and separately trained for each of the cases. A series of kicks, both static and in motion, using two different gaits were developed. The kicks obtained show very high reliability and, when compared to state of the art kicks, displayed a very high time performance improvement. This opens the door to more dynamic games with faster kicks in the RoboCup simulation 3D league.

show abstract

Section: Action Spacesupporting

confidence: 73%

“…This forward motion corresponds to one of several skills developed for usage in a game environment. These correspond to a sprinting and running pace [12].…”

Section: Motivationmentioning

confidence: 99%

“…This component is comprised of the skills that generate the motion from which the kick follows. These skills were previously developed by Miguel Abreu using a PPO algorithm [12].…”

Section: Skillsmentioning

confidence: 99%

Section: Observation Spacementioning

confidence: 99%

See 2 more Smart Citations

Humanoid Robot Kick in Motion Ability for Playing Robotic Soccer

Teixeira

Silva

Abreu

et al. 2020

2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Each oscillator has a set of parameters that are generally tuned by some trail-intensive method like trials and error, machine learning (ML) algorithms or both of them [25,33]. Some other approaches in this class are designed based on learning from scratch and mostly are based on Reinforcement Learning (RL) algorithms [1,19] which need many samples to be able to generate walking that takes a considerable amount of time. Unlike the model-free approaches, the fundamental core of the model-based approach is a dynamics model of the robot.…”

Section: Introductionmentioning

confidence: 99%

A modular framework to generate robust biped locomotion: from planning to control

et al. 2021

Self Cite

View full text Add to dashboard Cite

Biped robots are inherently unstable because of their complex kinematics as well as dynamics. Despite many research efforts in developing biped locomotion, the performance of biped locomotion is still far from the expectations. This paper proposes a model-based framework to generate stable biped locomotion. The core of this framework is an abstract dynamics model which is composed of three masses to consider the dynamics of stance leg, torso, and swing leg for minimizing the tracking problems. According to this dynamics model, we propose a modular walking reference trajectories planner which takes into account obstacles to plan all the references. Moreover, this dynamics model is used to formulate the controller as a Model Predictive Control (MPC) scheme which can consider some constraints in the states of the system, inputs, outputs, and also mixed input-output. The performance and the robustness of the proposed framework are validated by performing several numerical simulations using MATLAB. Moreover, the framework is deployed on a simulated torque-controlled humanoid to verify its performance and robustness. The simulation results show that the proposed framework is capable of generating biped locomotion robustly.

show abstract