CPG Driven RBF Network Control with Reinforcement Learning for Gait Optimization of a Dung Beetle-Like Robot

Pitchai, Matheshwaran; Xiong, Xiaofeng; Thor, Mathias; Billeschou, Peter; Mailänder, Peter Lukas; Leung, Binggwong; Kulvičius, Tomas; Manoonpong, Poramate

doi:10.1007/978-3-030-30487-4_53

Cited by 20 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a significant body of research into learning using non-spiking neurons which investigates closed-loop feedback control and online adaptation of legged robot locomotion. Thor and Manoonpong ( 2019 ) and Pitchai et al ( 2019 ) use learning to find efficient parameters within nCPGs. The controllers are able to adapt amplitude, frequency, and phase of nCPGs within the network.…”

Section: Introductionmentioning

confidence: 99%

Flexible Spiking CPGs for Online Manipulation During Hexapod Walking

2020

Self Cite

View full text Add to dashboard Cite

Neural signals for locomotion are influenced both by the neural network architecture and sensory inputs coordinating and adapting the gait to the environment. Adaptation relies on the ability to change amplitude, frequency, and phase of the signals within the sensorimotor loop in response to external stimuli. However, in order to experiment with closed-loop control, we first need a better understanding of the dynamics of the system and how adaptation works. Based on insights from biology, we developed a spiking neural network capable of continuously changing amplitude, frequency, and phase online. The resulting network is deployed on a hexapod robot in order to observe the walking behavior. The morphology and parameters of the network results in a tripod gait, demonstrating that a design without afferent feedback is sufficient to maintain a stable gait. This is comparable to results from biology showing that deafferented samples exhibit a tripod-like gait and adds to the evidence for a meaningful role of network topology in locomotion. Further, this work enables research into the role of sensory feedback and high-level control signals in the adaptation of gait types. A better understanding of the neural control of locomotion relates back to biology where it can provide evidence for theories that are currently not testable on live insects.

show abstract

Section: Introductionmentioning

confidence: 99%

Flexible Spiking CPGs for Online Manipulation During Hexapod Walking

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The sensory event phase estimation is utilized by the RBF neuron, which learns to anticipate the sensory event, when x ( t ) ≈ 1. The RBF neuron activity coupled to the CPG represents a particular phase interval, be it motor phase (Pitchai et al, 2019 ) or sensory phase. The RBF neuron uses the activity function…”

Section: The Gait Locomotion Controllermentioning

confidence: 99%

Self-Learning Event Mistiming Detector Based on Central Pattern Generator

2021

View full text Add to dashboard Cite

A repetitive movement pattern of many animals, a gait, is controlled by the Central Pattern Generator (CPG), providing rhythmic control synchronous to the sensed environment. As a rhythmic signal generator, the CPG can control the motion phase of biomimetic legged robots without feedback. The CPG can also act in sensory synchronization, where it can be utilized as a sensory phase estimator. Direct use of the CPG as the estimator is not common, and there is little research done on its utilization in the phase estimation. Generally, the sensory estimation augments the sensory feedback information, and motion irregularities can reveal from comparing measurements with the estimation. In this work, we study the CPG in the context of phase irregularity detection, where the timing of sensory events is disturbed. We propose a novel self-supervised method for learning mistiming detection, where the neural detector is trained by dynamic Hebbian-like rules during the robot walking. The proposed detector is composed of three neural components: (i) the CPG providing phase estimation, (ii) Radial Basis Function neuron anticipating the sensory event, and (iii) Leaky Integrate-and-Fire neuron detecting the sensory mistiming. The detector is integrated with the CPG-based gait controller. The mistiming detection triggers two reflexes: the elevator reflex, which avoids an obstacle, and the search reflex, which grasps a missing foothold. The proposed controller is deployed and trained on a hexapod walking robot to demonstrate the mistiming detection in real locomotion. The trained system has been examined in the controlled laboratory experiment and real field deployment in the Bull Rock cave system, where the robot utilized mistiming detection to negotiate the unstructured and slippery subterranean environment.

show abstract

“…ANNs have been shown to effectively manipulate amplitude, frequency, and phase in legged robots to create adaptive controllers (Nachstedt et al, 2013 ; Schilling et al, 2013 ; Barikhan et al, 2014 ; Dürr et al, 2019 ; Pitchai et al, 2019 ; Thor and Manoonpong, 2019 ). Thor and Manoonpong ( 2019 ) used an error signal to update synaptic weights for adaptation of frequency to optimize walking, resulting in increased efficiency and reduced tracking error.…”

Section: Introductionmentioning

confidence: 99%

Integrating Non-spiking Interneurons in Spiking Neural Networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

Researchers working with neural networks have historically focused on either non-spiking neurons tractable for running on computers or more biologically plausible spiking neurons typically requiring special hardware. However, in nature homogeneous networks of neurons do not exist. Instead, spiking and non-spiking neurons cooperate, each bringing a different set of advantages. A well-researched biological example of such a mixed network is a sensorimotor pathway, responsible for mapping sensory inputs to behavioral changes. This type of pathway is also well-researched in robotics where it is applied to achieve closed-loop operation of legged robots by adapting amplitude, frequency, and phase of the motor output. In this paper we investigate how spiking and non-spiking neurons can be combined to create a sensorimotor neuron pathway capable of shaping network output based on analog input. We propose sub-threshold operation of an existing spiking neuron model to create a non-spiking neuron able to interpret analog information and communicate with spiking neurons. The validity of this methodology is confirmed through a simulation of a closed-loop amplitude regulating network inspired by the internal feedback loops found in insects for posturing. Additionally, we show that non-spiking neurons can effectively manipulate post-synaptic spiking neurons in an event-based architecture. The ability to work with mixed networks provides an opportunity for researchers to investigate new network architectures for adaptive controllers, potentially improving locomotion strategies of legged robots.

show abstract

CPG Driven RBF Network Control with Reinforcement Learning for Gait Optimization of a Dung Beetle-Like Robot

Cited by 20 publications

References 12 publications

Flexible Spiking CPGs for Online Manipulation During Hexapod Walking

Flexible Spiking CPGs for Online Manipulation During Hexapod Walking

Self-Learning Event Mistiming Detector Based on Central Pattern Generator

Integrating Non-spiking Interneurons in Spiking Neural Networks

Contact Info

Product

Resources

About