Where and When: Event-Based Spatiotemporal Trajectory Prediction from the iCub’s Point-Of-View

Monforte, Marco; Arriandiaga, Ander; Glover, Arren; Bartolozzi, Chiara

doi:10.1109/icra40945.2020.9197373

Cited by 3 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That seed could grow into a mature field thanks to the availability of hardware that could be more easily tuned by nonexperts with standard software tools [428] and of quality DVS [429] and neuromorphic computing chips and systems [412,430] featuring many instances of neurons and (learning) synapses that could be used as computational primitives for perception and decision making. Since then, neuromorphic robotics followed three main paths, with the development of visual perception for robots using event-driven (dynamic) vision sensors [375,431], proof-of-concept systems linking sensing to control [432] and SNN for the control of motors [433,434]. At the same time, the neurorobotics community started developing models of perception, cognition and behaviour based on SNN, with recent attempts to implement those on neuromorphic platforms [435][436][437].…”

Section: Statusmentioning

confidence: 99%

2022 roadmap on neuromorphic computing and engineering

Christensen

Dittmann

Linares-Barranco

et al. 2022

Neuromorph. Comput. Eng.

Self Cite

329

200

View full text Add to dashboard Cite

Modern computation based on the von Neumann architecture is today a mature cutting-edge science. In the Von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 1018 calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this Roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The Roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this Roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community.

show abstract

Section: Statusmentioning

confidence: 99%

2022 roadmap on neuromorphic computing and engineering

Christensen

Dittmann

Linares-Barranco

et al. 2022

Neuromorph. Comput. Eng.

Self Cite

329

200

View full text Add to dashboard Cite

show abstract

“…After an offline evaluation of the accuracy, online experiments show the quadrotor avoiding the ball under different scenarios. In [15] an event camera with an Encoder-Decoder [16] LSTM network predicts the giver's motion during a handover task from the iCub robot perspective. The pipeline predicts both spatial and temporal future points, allowing the robot to know in advance where to move, compensating for internal delays in the perception-action loop.…”

Section: Introductionmentioning

confidence: 99%

“…This suits the event-driven paradigm, for which the number of data points cannot be defined before the action unfolds. This "continuous memory" also allows feeding as input only the latest point to update the prediction at each time step, avoiding redundant computation in the network due to possible buffers [15]. To boost the learning process, and cope with the data-hungry nature of the chosen model-free approach, similarly to [10], we train on simulated trajectories, parametrized to capture the same statistics of the real ones, before fine-tuning the models in real-world experiments.…”

Section: Introductionmentioning

confidence: 99%

Exploiting Event Cameras for Spatio-Temporal Prediction of Fast-Changing Trajectories

Monforte

Arriandiaga

Glover

et al. 2020

2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS)

View full text Add to dashboard Cite

This paper investigates trajectory prediction for robotics, to improve the interaction of robots with moving targets, such as catching a bouncing ball. Unexpected, highly-non-linear trajectories cannot easily be predicted with regression-based fitting procedures, therefore we apply state of the art machine learning, specifically based on Long-Short Term Memory (LSTM) architectures. In addition, fast moving targets are better sensed using event cameras, which produce an asynchronous output triggered by spatial change, rather than at fixed temporal intervals as with traditional cameras. We investigate how LSTM models can be adapted for event camera data, and in particular look at the benefit of using asynchronously sampled data.

show abstract