An Event-by-Event Feature Detection and Tracking Invariant to Motion Direction and Velocity

Dardelet, Laurent; Benosman, Ryad; Ieng, Sio-Hoï

doi:10.36227/techrxiv.17013824

Cited by 4 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, we made a formal bridge between this event-based MLR and a SNN, demonstrating the bio-plausibility of this method and its possible integration to neuromorphic hardware. A serie of similar algorithms are applied to real-life applications such as optical flow [4], depth estimation [16], or gesture recognition [43]. Recently, such an algorithm was used in the CASPR mission on the International Space Station, illustrating, given the stringent selection processes used by the space sector, that such algorithms have reached maturity for industry.…”

Section: What Can Neuroscience Bring To Computer Vision?mentioning

confidence: 99%

“…Another main contribution is to provide a model that is suitable for learning any type of spatio-temporal spiking motifs and that can be trained in a supervised way by providing a dataset of supervision pairs. This allows for a more flexible definition of the model using this properly labeled dataset instead of relying on a careful description of the physical rules governing a task, e.g., the luminance conservation principle for motion detection [4,16].…”

Section: Synthesis and Main Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

Learning heterogeneous delays in a layer of spiking neurons for fast motion detection

Grimaldi,

Perrinet

2023

Biol Cybern

View full text Add to dashboard Cite

The response of a biological neuron depends on the precise timing of afferent spikes. This temporal aspect of the neuronal code is essential in understanding information processing in neurobiology and applies particularly well to the output of neuromorphic hardware such as eventbased cameras. However, most artificial neuronal models do not take advantage of this minute temporal dimension. Inspired by this neuroscientific observation, we develop a model for the efficient detection of temporal spiking motifs based on a layer of neurons with heterogeneous delays which we apply to the computer vision task of motion detection. Indeed, the variety of synaptic delays on the dendritic tree allows to synchronize synaptic inputs as they reach the basal dendritic tree. We show this can be formalized as a time-invariant logistic regression which can be trained using labeled data. We apply this model to solve the specific computer vision problem of motion detection, and demonstrate its application to synthetic naturalistic videos transformed into event streams similar to the output of event-based cameras. In particular, we quantify how the accuracy of the model can vary with the total computational load. This end-to-end event-driven computational brick could help improve the performance of future Spiking Neural Network (SNN) algorithms and their prospective use in neuromorphic chips.

show abstract

Section: What Can Neuroscience Bring To Computer Vision?mentioning

confidence: 99%

Section: Synthesis and Main Contributionsmentioning

confidence: 99%

Learning heterogeneous delays in a layer of spiking neurons for fast motion detection

Grimaldi,

Perrinet

2023

Biol Cybern

View full text Add to dashboard Cite

show abstract

“…The concepts of this algorithm were successfully used for quadrotor obstacle avoidance (Falanga, Kim and Scaramuzza, 2019). Feature tracking has also been studied using the estimation of velocity of non-specified features (Dardelet, Ieng and Benosman, 2018) and contour motion (Barranco, Fermuller and Aloimonos, 2014).…”

Section: Related Studiesmentioning

confidence: 99%

Event-based tracking of human hands

Duarte

Safeea

Neto

2021

View full text Add to dashboard Cite

Purpose This paper proposes a novel method for human hands tracking using data from an event camera. The event camera detects changes in brightness, measuring motion, with low latency, no motion blur, low power consumption and high dynamic range. Captured frames are analysed using lightweight algorithms reporting three-dimensional (3D) hand position data. The chosen pick-and-place scenario serves as an example input for collaborative human–robot interactions and in obstacle avoidance for human–robot safety applications. Design/methodology/approach Events data are pre-processed into intensity frames. The regions of interest (ROI) are defined through object edge event activity, reducing noise. ROI features are extracted for use in-depth perception. Findings Event-based tracking of human hand demonstrated feasible, in real time and at a low computational cost. The proposed ROI-finding method reduces noise from intensity images, achieving up to 89% of data reduction in relation to the original, while preserving the features. The depth estimation error in relation to ground truth (measured with wearables), measured using dynamic time warping and using a single event camera, is from 15 to 30 millimetres, depending on the plane it is measured. Originality/value Tracking of human hands in 3 D space using a single event camera data and lightweight algorithms to define ROI features (hands tracking in space).

show abstract

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Grimaldi

Gruel²,

Besnainou

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

Why do neurons communicate through spikes? By definition, spikes are all-or-none neural events which occur at continuous times. In other words, spikes are on one side binary, existing or not without further details, and on the other, can occur at any asynchronous time, without the need for a centralized clock. This stands in stark contrast to the analog representation of values and the discretized timing classically used in digital processing and at the base of modern-day neural networks. As neural systems almost systematically use this so-called event-based representation in the living world, a better understanding of this phenomenon remains a fundamental challenge in neurobiology in order to better interpret the profusion of recorded data. With the growing need for intelligent embedded systems, it also emerges as a new computing paradigm to enable the efficient operation of a new class of sensors and event-based computers, called neuromorphic, which could enable significant gains in computation time and energy consumption—a major societal issue in the era of the digital economy and global warming. In this review paper, we provide evidence from biology, theory and engineering that the precise timing of spikes plays a crucial role in our understanding of the efficiency of neural networks.

show abstract

An Event-by-Event Feature Detection and Tracking Invariant to Motion Direction and Velocity

Abstract: Contour velocity estimation and tracking from a fully event-based perspective.<br>

Cited by 4 publications

References 20 publications

Learning heterogeneous delays in a layer of spiking neurons for fast motion detection

Learning heterogeneous delays in a layer of spiking neurons for fast motion detection

Event-based tracking of human hands

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Contact Info

Product

Resources

About