What Can Neuromorphic Event-Driven Precise Timing Add to Spike-Based Pattern Recognition?

Abstract

This letter introduces a study to precisely measure what an increase in spike timing precision can add to spike-driven pattern recognition algorithms. The concept of generating spikes from images by converting gray levels into spike timings is currently at the basis of almost every spike-based modeling of biological visual systems. The use of images naturally leads to generating incorrect artificial and redundant spike timings and, more important, also contradicts biological findings indicating that visual processing is massively parallel, asynchronous with high temporal resolution. A new concept for acquiring visual information through pixel-individual asynchronous level-crossing sampling has been proposed in a recent generation of asynchronous neuromorphic visual sensors. Unlike conventional cameras, these sensors acquire data not at fixed points in time for the entire array but at fixed amplitude changes of their input, resulting optimally sparse in space and time-pixel individually and precisely timed only if new, (previously unknown) information is available (event based). This letter uses the high temporal resolution spiking output of neuromorphic event-based visual sensors to show that lowering time precision degrades performance on several recognition tasks specifically when reaching the conventional range of machine vision acquisition frequencies (30-60 Hz). The use of information theory to characterize separability between classes for each temporal resolution shows that high temporal acquisition provides up to 70% more information that conventional spikes generated from frame-based acquisition as used in standard artificial vision, thus drastically increasing the separability between classes of objects. Experiments on real data show that the amount of information loss is correlated with temporal precision. Our information-theoretic study highlights the potentials of neuromorphic asynchronous visual sensors for both practical applications and theoretical investigations. Moreover, it suggests that representing visual information as a precise sequence of spike times as reported in the retina offers considerable advantages for neuro-inspired visual computations.

A Motion-Based Feature for Event-Based Pattern Recognition

“…or asynchronous event-based (Akolkar et al, 2015) visual information. Works about natural image statistics (Hyvarinen et al, 2009) showed that similar decompositions of visual information emerge naturally from independent component analysis applied on patches collected on natural images.…”

“…s −1 , i.e., inversely close to the temporal precision of the visual events, estimated over 1 ms (cf. Akolkar et al, 2015). Motions with intensities less than v min are then discarded: they are assumed as belonging to static or faraway objects in the background visual scene.…”

“…Rogister et al, 2012; Carneiro et al, 2013; Camuñas-Mesa et al, 2014; Firouzi and Conradt, 2015), corner detection (Clady et al, 2015), or shape recognition (see Pérez-Carrasco et al, 2013; Akolkar et al, 2015; Orchard et al, 2015a,b; Lee et al, 2016). This strong interest in such a sensor is essentially due to its ability to provide visual information as a high temporal resolution, luminance-free, and non-redundant stream.…”

Inspired by nature: timescale-free and grid-free event-based computing with spiking neural networks

“…Other studies process the data stream on an event-by-event basis [16][17][18][19][20][21][22][23][24][25][26][27][28][29], with the timing of an event determining its relevance to an internal model. Some event-based studies accumulate event activities and then decay them as time passes [30][31][32][33][34].…”

“…• SNNs [34], with event-driven convolutional neural networks [8,31], or with leaky integrate-and-fire neurons [32,50] in deep belief networks [33] • Event-based iterative closest point [30,48] Continued on next page [26,35,39], for occlusion of multiple objects [83] • Self-organising map [60] • Spike-based vector integration to end point [25] Event by event, with a temporal gradient for each event…”