Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot

Schilling, Malte; Konen, Kai; Ohl, Frank W.; Korthals, Timo

doi:10.1109/iros45743.2020.9341754

Cited by 21 publications

(14 citation statements)

References 25 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, it could be a requirement to learn new motor patterns for new tasks. Motor pattern adaptation has been applied both to CPG-based locomotion controllers (Nakanishi et al, 2004 ; Oliveira et al, 2011 ) and deep neural network locomotion controllers (Clune et al, 2011 ; Hwangbo et al, 2019 ; Lee et al, 2020 ; Schilling et al, 2020a , b ; Yang et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Locomotion Control With Frequency and Motor Pattern Adaptations

Thor

Strohmer

Manoonpong

2021

Front. Neural Circuits

View full text Add to dashboard Cite

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

show abstract

Section: Introductionmentioning

confidence: 99%

Locomotion Control With Frequency and Motor Pattern Adaptations

Thor

Strohmer

Manoonpong

2021

Front. Neural Circuits

View full text Add to dashboard Cite

show abstract

“…We already used a decentralized control architecture in a DRL setup, in which for each leg an individual controller was trained that only got local information. This showed to be sufficient to learn stable walking behavior and-even more-it learned much faster compared to a centralized baseline approach and showed good generalization capabilities [92]. As a second principle, the focus of this article is on showing that and how such a decentralized control structure can successfully employ higher level planning realized as internal simulation.…”

Section: Discussionmentioning

confidence: 99%

From Adaptive Locomotion to Predictive Action Selection – Cognitive Control for a Six-Legged Walker

et al. 2022

Self Cite

View full text Add to dashboard Cite

Locomotion in animals provides a model for adaptive behavior as it is able to deal with various kinds of perturbations. Work in insects suggests that this evolved flexibility results from a modular architecture, which can be characterized by a recurrent neural network allowing for various emerging attractor states. Whereas a lower control-level coordinates joint movements on a short timescale, a higher-level handles action selection on longer timescales. Implementation of such a control system on a walking hexapod robot was able to deal with various walking patterns including disturbances such as uneven terrain or loss of a leg. Here, we propose a cognitive expansion to the adaptive control system that allows dealing with novel challenging situations. This approach makes use of an internal simulation-based planner that is triggered when the model-free controller fails to recover from an unstable pose. Using a grounded internal body model, the planner then tries, in internal simulation, different solutions out of context, and thus, proposes a new plan to be executed on the real robot. We demonstrate the feasibility of this control approach for walking over terrain with uncertain footholds in three scenarios.

show abstract

“…To date, different machine learning techniques have been actively explored for locomotion control. The techniques include (deep) reinforcement learning (RL) [165,217], imitation learning [218], intelligent trial and error [219], and evolutionary computation [166,220,221].…”

Section: Machine Learning-based Controlmentioning

confidence: 99%

“…The reinforcement learning method can automatically find motion planning policies for hexapod robots moving on uneven piles of plum-blossom (Figure 6b,c). Schilling et al [217] introduced a biologically-inspired decentralized control architecture with deep reinforcement learning for adaptive locomotion in a hexapod robot. The architecture consists of six neural control modules, each of which controls one leg (Figure 6a,c).…”

Section: Machine Learning-based Controlmentioning

confidence: 99%

Insect-Inspired Robots: Bridging Biological and Artificial Systems

Manoonpong

Patané

Xiong

et al. 2021

Sensors

View full text Add to dashboard Cite

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

show abstract

Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot

Cited by 21 publications

References 25 publications

Locomotion Control With Frequency and Motor Pattern Adaptations

Locomotion Control With Frequency and Motor Pattern Adaptations

From Adaptive Locomotion to Predictive Action Selection – Cognitive Control for a Six-Legged Walker

Insect-Inspired Robots: Bridging Biological and Artificial Systems

Contact Info

Product

Resources

About