<i>Learning to Walk</i>: Bio-Mimetic Hexapod Locomotion via Reinforcement-Based Spiking Central Pattern Generation

Lele, Ashwin Sanjay; Fang, Yan; Ting, Justin; Raychowdhury, Arijit

doi:10.1109/jetcas.2020.3033135

Cited by 15 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, it has emerged as a powerful technique widely applied in the field of robotics. Its applications encompass gait training [52,53], motion strategy learning [54], trajectory optimisation [55], automatic residual learning [56,57], and optimisation control [58], among others. Through the use of reinforcement learning, robots can continuously learn and improve their control strategies and behaviors by actively interacting with the environment.…”

Section: Introductionmentioning

confidence: 99%

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

Liu,

Rong,

Neri

et al. 2024

ICA

View full text Add to dashboard Cite

In this paper, we propose a novel Reinforcement Learning (RL) algorithm for robotic motion control, that is, a constrained Deep Deterministic Policy Gradient (DDPG) deviation learning strategy to assist biped robots in walking safely and accurately. The previous research on this topic highlighted the limitations in the controller’s ability to accurately track foot placement on discrete terrains and the lack of consideration for safety concerns. In this study, we address these challenges by focusing on ensuring the overall system’s safety. To begin with, we tackle the inverse kinematics problem by introducing constraints to the damping least squares method. This enhancement not only addresses singularity issues but also guarantees safe ranges for joint angles, thus ensuring the stability and reliability of the system. Based on this, we propose the adoption of the constrained DDPG method to correct controller deviations. In constrained DDPG, we incorporate a constraint layer into the Actor network, incorporating joint deviations as state inputs. By conducting offline training within the range of safe angles, it serves as a deviation corrector. Lastly, we validate the effectiveness of our proposed approach by conducting dynamic simulations using the CRANE biped robot. Through comprehensive assessments, including singularity analysis, constraint effectiveness evaluation, and walking experiments on discrete terrains, we demonstrate the superiority and practicality of our approach in enhancing walking performance while ensuring safety. Overall, our research contributes to the advancement of biped robot locomotion by addressing gait optimisation from multiple perspectives, including singularity handling, safety constraints, and deviation learning.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

Liu,

Rong,

Neri

et al. 2024

ICA

View full text Add to dashboard Cite

show abstract

“…To meet control demands and ensure that the robot's speed, displacement, foot placement and other variables of the robot are consistent with the desired trajectory, it is necessary to further study the motion-tracking control of the joint. A variety of control methods have been developed to address the motion trajectory tracking problem, such as PD control [13,14,15], fuzzy control [16,17,18], neural network [19,20,21,22,23,24], model predictive control (MPC) [25,26], human-simulated intelligent control (HSIC) [27,28] or a variety of combination methods [29,30,31,32,33].…”

Section: Introductionmentioning

confidence: 99%

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

Liu

Zhang

Mastoi

et al. 2023

ICA

View full text Add to dashboard Cite

To guarantee their locomotion, biped robots need to walk stably. The latter is achieved by a high performance in joint control. This article addresses this issue by proposing a novel human-simulated fuzzy (HF) membrane control system of the joint angles. The proposed control system, human-simulated fuzzy membrane controller (HFMC), contains several key elements. The first is an HF algorithm based on human-simulated intelligent control (HSIC). This HF algorithm incorporates elements of both multi-mode proportional-derivative (PD) and fuzzy control, aiming at solving the chattering problem of multi-mode switching while improving control accuracy. The second is a membrane architecture that makes use of the natural parallelisation potential of membrane computing to improve the real-time performance of the controller. The proposed HFMC is utilised as the joint controller for a biped robot. Numerical tests in a simulation are carried out with the planar and slope walking of a five-link biped robot, and the effectiveness of the HFMC is verified by comparing and evaluating the results of the designed HFMC, HSIC and PD. Experimental results demonstrate that the proposed HFMC not only retains the advantages of traditional PD control but also improves control accuracy, real-time performance and stability.

show abstract

“…Decentralized control strategies can be embedded in a RL approach, to adapt to different types of articulated robots [30]. The combination of CPG and RL provides a breakthrough in end-to-end learning for mobile robots [31]. Mimicking the brain of an insect, a hexapod robot can also have complex advanced cognitive control.…”

Section: Introductionmentioning

confidence: 99%

A Soft Actor-Critic Approach for a Blind Walking Hexapod Robot with Obstacle Avoidance

Wang,

Li,

Huangfu

et al. 2023

Actuators

View full text Add to dashboard Cite

This paper investigates a path planning approach for the walking and obstacle avoidance of a blind hexapod robot in various field conditions. Hexapod robots often perform field tasks in unstructured environments, and their external sensors are affected by weather and light. This paper proposes the use of internal sensors to sense the terrain and a slightly modified soft actor-critic algorithm to train the motion strategy. A hexapod robot is capable of walking smoothly on rough ground only using internal sensors that are not affected by weather factors, and the soft actor-critic approach is superior for overcoming high-dimensional issues for multi-degree-freedom robot motion in unstructured environments.The experiments showed that the hexapod robot not only traversed rugged terrain at a fixed speed but also possessed obstacle avoidance capabilities.

show abstract

Learning to Walk: Bio-Mimetic Hexapod Locomotion via Reinforcement-Based Spiking Central Pattern Generation

Cited by 15 publications

References 26 publications

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

A Soft Actor-Critic Approach for a Blind Walking Hexapod Robot with Obstacle Avoidance

Contact Info

Product

Resources

About