Towards Grasping with Spiking Neural Networks for Anthropomorphic Robot Hands

Tieck, J. Camilo Vasquez; Donat, Heiko; Kaiser, Jacques; Peric, Igor; Ulbrich, Stefan; Roennau, Arne; Zöllner, Marius; Dillmann, Rüdiger

doi:10.1007/978-3-319-68600-4_6

Cited by 27 publications

(21 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This would enable the setup to be extended to temporal sequence learning and reinforcement learning tasks [24], [25]. Additionally, other components of the grasp-type recognition experiment could be implemented with spiking networks, such as reaching motions [26], [27], grasping motions [28] and depth perception [16]. This work paves the way towards the integration of brain-inspired computational paradigms into the field of robotics.…”

Section: Discussionmentioning

confidence: 99%

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Kaiser

Friedrich

Tieck

et al. 2020

2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)

Self Cite

View full text Add to dashboard Cite

Spike-based communication between biological neurons is sparse and unreliable. This enables the brain to process visual information from the eyes efficiently. Taking inspiration from biology, artificial spiking neural networks coupled with silicon retinas attempt to model these computations. Recent findings in machine learning allowed the derivation of a family of powerful synaptic plasticity rules approximating backpropagation for spiking networks. Are these rules capable of processing real-world visual sensory data? In this paper, we evaluate the performance of Event-Driven Random Back-Propagation (eRBP) at learning representations from event streams provided by a Dynamic Vision Sensor (DVS). First, we show that eRBP matches state-of-the-art performance on the DvsGesture dataset with the addition of a simple covert attention mechanism. By remapping visual receptive fields relatively to the center of the motion, this attention mechanism provides translation invariance at low computational cost compared to convolutions. Second, we successfully integrate eRBP in a real robotic setup, where a robotic arm grasps objects according to detected visual affordances. In this setup, visual information is actively sensed by a DVS mounted on a robotic head performing microsaccadic eye movements. We show that our method classifies affordances within 100ms after microsaccade onset, which is comparable to human performance reported in behavioral study. Our results suggest that advances in neuromorphic technology and plasticity rules enable the development of autonomous robots operating at high speed and low energy consumption.

show abstract

Section: Discussionmentioning

confidence: 99%

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Kaiser

Friedrich

Tieck

et al. 2020

2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The motion of the hand is also modelled with a motor primitive, but instead of controlling joints, it controls the activation parameters of the finger primitives. The hand primitives are organized in a hierarchy coordinating the finger primitives as in [27], [29]. The hand primitives represent different grasping affordances -sphere, cylinder and pinch according to [13] and rest position.…”

Section: B Hand Primitives and Hierarchymentioning

confidence: 99%

“…Although, there are other approaches using SNN for motion control using force feedback, to the best of our knowledge, there is no implementation of an SNN for soft grasping with a 5-finger anthropomorphic hand performing compliant control without force sensors using the standart joint interface. We can model a system using SNN for soft-grasping using an anthropomorphic robotic hand taking inspiration from biology and using the principles presented in previous work for a hierarchy of motor primitives with SNN to model the hand [27], to model finger reflexes [28], to coordinate multiple primitives [29], and to combine activation modalities [30].…”

Section: Introductionmentioning

confidence: 99%

Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons

Tieck

Secker

Kaiser

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Evolution gave humans advanced grasping capabilities combining an adaptive hand with efficient control. Grasping motions can quickly be adapted if the object moves or deforms. Soft-grasping with an anthropomorphic hand is a great capability for robots interacting with objects shaped for humans. Nevertheless, most robotic applications use vacuum, 2-finger or custom made grippers. We present a biologically inspired spiking neural network (SNN) for soft-grasping to control a robotic hand. Two control loops are combined, one from motor primitives and one from a compliant controller activated by a reflex. The finger primitives represent synergies between joints and hand primitives represent different affordances. Contact is detected with a mechanism based on inter-neuron circuits in the spinal cord to trigger reflexes. A Schunk SVH 5-finger hand was used to grasp objects with different shapes, stiffness and sizes. The SNN adapted the grasping motions without knowing the exact properties of the objects. The compliant controller with online learning proved to be sensitive, allowing even the grasping of balloons. In contrast to deep learning approaches, our SNN requires one example of each grasping motion to train the primitives. Computation of the inverse kinematics or complex contact point planning is not required. This approach simplifies the control and can be used on different robots providing similar adaptive features as a human hand. A physical imitation of a biological system implemented completely with SNN and a robotic hand can provide new insights into grasping mechanisms.

show abstract

“…Recently, SNNs have gained popularity in the robotics community [6]- [13], due to potential advantages over ANNs including low power consumption and low latencies. The community also has access to a growing neuromorphic hardware set, such as Intel's Loihi [2].…”

Section: Introductionmentioning

confidence: 99%

“…The community also has access to a growing neuromorphic hardware set, such as Intel's Loihi [2]. SNNs have been used in disciplines including robot grasping [6], robot navigation [7], and motion planning [8]. They are however not yet widely…”

Section: Introductionmentioning

confidence: 99%

Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments

Hussaini,

Milford,

Fischer

2021

Preprint

View full text Add to dashboard Cite

Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies, and challenges including the non-differentiable nature of event spikes. Much of the initial research in this area has converted deep neural networks to equivalent SNNs, but this conversion approach potentially negates some of the potential advantages of SNN-based approaches developed from scratch. One promising area for high performance SNNs is template matching and image recognition. This research introduces the first high performance SNN for the Visual Place Recognition (VPR) task: given a query image, the SNN has to find the closest match out of a list of reference images. At the core of this new system is a novel assignment scheme that implements a form of ambiguity-informed salience, by up-weighting single-placeencoding neurons and down-weighting "ambiguous" neurons that respond to multiple different reference places. In a range of experiments on the challenging Oxford RobotCar and Nordland datasets, we show that our SNN achieves comparable VPR performance to state-of-the-art and classical techniques, and degrades gracefully in performance with an increasing number of reference places. Our results provide a significant milestone towards SNNs that can provide robust, energy-efficient and low latency robot localization.

show abstract

Towards Grasping with Spiking Neural Networks for Anthropomorphic Robot Hands

Cited by 27 publications

References 15 publications

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons

Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments

Contact Info

Product

Resources

About