Microsaccades for Neuromorphic Stereo Vision

Kaiser, Jacques; Weinland, Jakob; Keller, Philip; Steffen, Lea; Tieck, J. Camilo Vasquez; Reichard, Daniel; Roennau, Arne; Conradt, Jörg; Dillmann, Rüdiger

doi:10.1007/978-3-030-01418-6_24

Cited by 9 publications

(15 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To exploit the advantages of an approach based on SNN and EBS, the implementations of event-based cooperative algorithms is compatible with neuromorphic hardware. In the case of Dikov et al (2017) and Kaiser et al (2018) this is SpiNNaker.…”

Section: Event-driven Stereoscopymentioning

confidence: 93%

“…This mechanism ensures that the integrating neuron is only capable of spiking if both blocker neurons of the ensemble are inhibited. Hence the integrating neuron only emits a pulse if both retina neurons are spiking (Dikov et al, 2017; Kaiser et al, 2018). In Dikov et al (2017) and Osswald et al (2017) disparities are calculated merely from dynamic scenes.…”

Section: Event-driven Stereoscopymentioning

confidence: 99%

“…This is simply to the fundamental technology of event-based cameras that perceive only changes in lighting and are thus not perceiving static scenes continuously. In Kaiser et al (2018), this approach, including EBS, is extended to static scenes by applying synchronous microsaccades. In biology, microsaccades are extremely fast and very small eye movements with low amplitude.…”

Section: Event-driven Stereoscopymentioning

confidence: 99%

“…The retina coordinates are visualized as light blue squares, the blocking neurons as blue circles and the collecting neuron as a red circle. Reprinted by permission from Springer Artificial Neural Networks and Machine Learning (Kaiser et al, 2018).…”

Section: Event-driven Stereoscopymentioning

confidence: 99%

See 3 more Smart Citations

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

et al. 2019

Self Cite

View full text Add to dashboard Cite

Any visual sensor, whether artificial or biological, maps the 3D-world on a 2D-representation. The missing dimension is depth and most species use stereo vision to recover it. Stereo vision implies multiple perspectives and matching, hence it obtains depth from a pair of images. Algorithms for stereo vision are also used prosperously in robotics. Although, biological systems seem to compute disparities effortless, artificial methods suffer from high energy demands and latency. The crucial part is the correspondence problem; finding the matching points of two images. The development of event-based cameras, inspired by the retina, enables the exploitation of an additional physical constraint—time. Due to their asynchronous course of operation, considering the precise occurrence of spikes, Spiking Neural Networks take advantage of this constraint. In this work, we investigate sensors and algorithms for event-based stereo vision leading to more biologically plausible robots. Hereby, we focus mainly on binocular stereo vision.

show abstract

Section: Event-driven Stereoscopymentioning

confidence: 93%

Section: Event-driven Stereoscopymentioning

confidence: 99%

Section: Event-driven Stereoscopymentioning

confidence: 99%

Section: Event-driven Stereoscopymentioning

confidence: 99%

See 2 more Smart Citations

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…To incorporate visual information, we consider event-based cameras a natural match for SNN. An stereo even-based camera setup [45] can be used to get the target point is 3D space for pre-grasping [46] and use micro-saccades [47] to detect the type of object and identify which grasping affordance to use. To integrate arm motion, an arm controller as in [48] can be incorporated to do visual servoing, or as in [35] to have primitives reach different points on a surface.The combination of visual information and arm motion, together with soft-grasping can achieve a more natural grasping process from recognition of the object to positioning the arm to grasping.…”

Section: Parameters Finger Primitivesmentioning

confidence: 99%

Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons

Tieck

Secker

Kaiser

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Evolution gave humans advanced grasping capabilities combining an adaptive hand with efficient control. Grasping motions can quickly be adapted if the object moves or deforms. Soft-grasping with an anthropomorphic hand is a great capability for robots interacting with objects shaped for humans. Nevertheless, most robotic applications use vacuum, 2-finger or custom made grippers. We present a biologically inspired spiking neural network (SNN) for soft-grasping to control a robotic hand. Two control loops are combined, one from motor primitives and one from a compliant controller activated by a reflex. The finger primitives represent synergies between joints and hand primitives represent different affordances. Contact is detected with a mechanism based on inter-neuron circuits in the spinal cord to trigger reflexes. A Schunk SVH 5-finger hand was used to grasp objects with different shapes, stiffness and sizes. The SNN adapted the grasping motions without knowing the exact properties of the objects. The compliant controller with online learning proved to be sensitive, allowing even the grasping of balloons. In contrast to deep learning approaches, our SNN requires one example of each grasping motion to train the primitives. Computation of the inverse kinematics or complex contact point planning is not required. This approach simplifies the control and can be used on different robots providing similar adaptive features as a human hand. A physical imitation of a biological system implemented completely with SNN and a robotic hand can provide new insights into grasping mechanisms.

show abstract

Biomorphic robot controls: event driven model free deep SNNs for complex visuomotor tasks

Dillmann

Rönnau

2022

Artif Life Robotics

View full text Add to dashboard Cite

Microsaccades for Neuromorphic Stereo Vision

Cited by 9 publications

References 16 publications

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons

Biomorphic robot controls: event driven model free deep SNNs for complex visuomotor tasks

Contact Info

Product

Resources

About