Event-Based Trajectory Prediction Using Spiking Neural Networks

Débat, Guillaume; Chauhan, Tushar; Cottereau, Benoit R.; Masquelier, Timothée; Paindavoine, Michel; Baurès, Robin

doi:10.3389/fncom.2021.658764

Cited by 12 publications

(9 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, the predictions obtained from the output spiking activity of our network were restricted to the pattern of optic flow (i.e., to the direction of self-motion). Future improvements could include an estimation of the exact displacement and velocities of the camera within the 3D environment (see for example (Debat et al, 2021) for an estimation of the 2D trajectories from the outputs of an SNN trained with STDP). This could be realized by adding other layers to the SNN or including a second event-based camera to support stereoscopic vision.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Event-based Extraction of Navigation Features from Unsupervised Learning of Optic Flow Patterns

Fricker

Chauhan

Hurter

et al. 2022

Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

Self Cite

View full text Add to dashboard Cite

We developed a Spiking Neural Network composed of two layers that processes event-based data captured by a dynamic vision sensor during navigation conditions. The training of the network was performed using a biologically plausible and unsupervised learning rule, Spike-Timing-Dependent Plasticity. With such an approach, neurons in the network naturally become selective to different components of optic flow, and a simple classifier is able to predict self-motion properties from the neural population output spiking activity. Our network has a simple architecture and a restricted number of neurons. Therefore, it is easy to implement on a neuromorphic chip and could be used for embedded applications necessitating low energy consumption.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This network was nonetheless complex and comprised different data formatting approaches distributed across multiple layers and neurons. Even more recently, (Debat et al, 2021) used the same type of SNN to show that learning through STDP led to neural populations whose spiking activity can be used to predict trajectories.…”

Section: Related Workmentioning

confidence: 99%

Event-based Extraction of Navigation Features from Unsupervised Learning of Optic Flow Patterns

Fricker

Chauhan

Hurter

et al. 2022

Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

Self Cite

View full text Add to dashboard Cite

show abstract

“…Debat et al, in their study published in 2021, processed the event-based camera data with a spiking neural network to estimate the trajectory of a ball [23]. In the study, the data was processed with a three-layer network with a Leaky Integrate and Fire (LIF) neuron model.…”

Section: Spiking Neural Network Solutionsmentioning

confidence: 99%

An Artificial Neural SLAM Framework for Event-Based Vision

Gelen

Atasoy

2023

IEEE Access

View full text Add to dashboard Cite

The SLAM problem for autonomous robots can be greatly improved by using event-based cameras. Compared to others, event-based cameras consume very low power while providing great temporal resolution and dynamic range. In this study, we propose a convolutional neural SLAM framework based solely on the event data. Event-based cameras generate events for pixels whose brightness changes. Therefore, the event data is rich in motion and edge information. The purpose of the proposed framework is to make all estimations using encoded information in event data. The proposed solution is in the form of keyframe-based visual SLAM, consisting of three neural networks that can estimate the relative camera pose, log-depth and features for loop closure detection. In the study, network architectures and learning curves for the trained networks are presented and it is shown that networks can learn the problems successfully. The proposed method has been developed and tested on a new dataset generated by the CARLA simulator. It has been shown that the proposed method is a SLAM solution and it can keep global drift under control with loop closure estimations. Evaluation metrics for estimations, evaluation of the global model and an analysis of run-time performance are also presented. Event-based cameras, convolutional neural networks, neural slam, visual slam. INDEX TERMS

show abstract

“…Thanks to the spike-based representation of information, the parallelism of multiple processing elements and their colocalization with memory units, neuromorphic systems achieve low-latency and lowpower performance, for which they have emerged as a worthy opponent for von Neumann architectures in computing systems [23]. These features also make them an appealing candidate for implantable neural interfaces, as their structure is inherently suited to simulating spiking neural networks, that are a powerful tool to solve problems of spatiotemporal pattern recognition [24][25][26][27]. As such, neuromorphic systems have already found successful application for processing a broad range of biological electrical signals [28][29][30][31][32][33], including high-frequency oscillations as pathological biomarkers of epilepsy [34][35][36] and seizure detection [37].…”

Section: Introductionmentioning

confidence: 99%

NET-TEN: a silicon neuromorphic network for low-latency detection of seizures in local field potentials

et al. 2023

View full text Add to dashboard Cite

Objective: Therapeutic intervention in neurological disorders still relies heavily on pharmacological solutions, while the treatment of patients with drug resistance remains an unresolved issue. This is particularly true for patients with epilepsy, 30% of whom are refractory to medications. Implantable devices for chronic recording and electrical modulation of brain activity have proved a viable alternative in such cases. To operate, the device should detect the relevant electrographic biomarkers from Local Field Potentials (LFPs) and determine the right time for stimulation. To enable timely interventions, the ideal device should attain biomarker detection with low latency while operating under low power consumption to prolong battery life. Approach: Here we introduce a fully-analog neuromorphic device implemented in CMOS technology for analyzing LFP signals in an in vitro model of acute ictogenesis. Neuromorphic networks have progressively gained a reputation as low-latency low-power computing systems, which makes them a promising candidate as processing core of next-generation implantable neural interfaces.Main results: The developed system can detect ictal and interictal events with ms-latency and with high precision, consuming on average 3.50 nW during the task.Significance: The work presented in this paper paves the way to a new generation of brain implantable devices for personalized closed-loop stimulation for epilepsy treatment.

show abstract

Event-Based Trajectory Prediction Using Spiking Neural Networks

Cited by 12 publications

References 48 publications

Event-based Extraction of Navigation Features from Unsupervised Learning of Optic Flow Patterns

Event-based Extraction of Navigation Features from Unsupervised Learning of Optic Flow Patterns

An Artificial Neural SLAM Framework for Event-Based Vision

NET-TEN: a silicon neuromorphic network for low-latency detection of seizures in local field potentials

Contact Info

Product

Resources

About