Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Chen, Shoushun; Akselrod, Polina; Bo, Zhao; Pérez-Carrasco, Jose Antonio; Linares-Barranco, B.; Culurciello, Eugenio

doi:10.1109/tpami.2011.120

Cited by 46 publications

(14 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many event-driven classification algorithms have been proposed to solve the classification task of the DVS recordings (Chen et al, 2012; Zhao et al, 2015; Peng et al, 2016). Here the classification results using two state-of-the-art algorithms are briefly presented to provide an initial classification benchmark.…”

Section: Resultsmentioning

confidence: 99%

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

et al. 2017

View full text Add to dashboard Cite

Neuromorphic vision research requires high-quality and appropriately challenging event-stream datasets to support continuous improvement of algorithms and methods. However, creating event-stream datasets is a time-consuming task, which needs to be recorded using the neuromorphic cameras. Currently, there are limited event-stream datasets available. In this work, by utilizing the popular computer vision dataset CIFAR-10, we converted 10,000 frame-based images into 10,000 event streams using a dynamic vision sensor (DVS), providing an event-stream dataset of intermediate difficulty in 10 different classes, named as “CIFAR10-DVS.” The conversion of event-stream dataset was implemented by a repeated closed-loop smooth (RCLS) movement of frame-based images. Unlike the conversion of frame-based images by moving the camera, the image movement is more realistic in respect of its practical applications. The repeated closed-loop image movement generates rich local intensity changes in continuous time which are quantized by each pixel of the DVS camera to generate events. Furthermore, a performance benchmark in event-driven object classification is provided based on state-of-the-art classification algorithms. This work provides a large event-stream dataset and an initial benchmark for comparison, which may boost algorithm developments in even-driven pattern recognition and object classification.

show abstract

Section: Resultsmentioning

confidence: 99%

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Our method with the average combination way runs twice as faster than the BoE statistical method during training on the MNIST-DVS and Posture-DVS datasets. When compared to other spiking convolutional neural networks [13], [14], our method runs hundreds to thousands of times faster during both training and classification. The large processing time consumptions in those previous methods prevent them from efficient online learning.…”

Section: Resultsmentioning

confidence: 99%

“…However, such shallow structures can only be used to recognize and track objects with simple shapes (e.g., oriented lines, squares, triangles, circles and ellipse) under constrained environment. Some other works inspired by the cortex visual processing model [18] extend to a deeper neuromorphic structure with more spiking convolution layers and learning based classifiers [13]–[17]. More convolution layers allow for extracting more complex visual features, and those classifiers with learning capability can be trained to recognize objects with complicated shape or texture patterns in natural environment.…”

Section: Introductionmentioning

confidence: 99%

Exploiting Lightweight Statistical Learning for Event-Based Vision Processing

Shi

Wang

et al. 2018

IEEE Access

View full text Add to dashboard Cite

This paper presents a lightweight statistical learning framework potentially suitable for low-cost event-based vision systems, where visual information is captured by a dynamic vision sensor (DVS) and represented as an asynchronous stream of pixel addresses (events) indicating a relative intensity change on those locations. A simple random ferns classifier based on randomly selected patch-based binary features is employed to categorize pixel event flows. Our experimental results demonstrate that compared to existing event-based processing algorithms, such as spiking convolutional neural networks (SCNNs) and the state-of-the-art bag-of-events (BoE)-based statistical algorithms, our framework excels in high processing speed (2× faster than the BoE statistical methods and >100× faster than previous SCNNs in training speed) with extremely simple online learning process, and achieves state-of-the-art classification accuracy on four popular address-event representation data sets: MNIST-DVS, Poker-DVS, Posture-DVS, and CIFAR10-DVS. Hardware estimation shows that our algorithm will be preferable for low-cost embedded system implementations.

show abstract

“…Recent advances in VLSI technology have solved the spiking neurons' implementation issues (Painkras et al, 2013; Merolla et al, 2014) that we faced previously because of their higher computational complexity compared to conventional artificial neurons, and thus have largely boosted the research and development of SNN. SNN has been used for many tasks such as learning (Ponulak and Kasiński, 2010; Yu et al, 2013a,b) and classification (Chen et al, 2012; Hu et al, 2013; Zhao et al, 2015). Among the various spiking neuron models proposed in the literatures (Izhikevich, 2003; Gütig and Sompolinsky, 2006), the most popular one is Leaky Integrate-and-Fire (LIF) neuron model (Burkitt, 2006a) due to its simplicity.…”

Section: Introductionmentioning

confidence: 99%

Hough Transform Implementation For Event-Based Systems: Concepts and Challenges

Seifozzakerini

Yau

Mao

et al. 2018

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Hough transform (HT) is one of the most well-known techniques in computer vision that has been the basis of many practical image processing algorithms. HT however is designed to work for frame-based systems such as conventional digital cameras. Recently, event-based systems such as Dynamic Vision Sensor (DVS) cameras, has become popular among researchers. Event-based cameras have a significantly high temporal resolution (1 μs), but each pixel can only detect change and not color. As such, the conventional image processing algorithms cannot be readily applied to event-based output streams. Therefore, it is necessary to adapt the conventional image processing algorithms for event-based cameras. This paper provides a systematic explanation, starting from extending conventional HT to 3D HT, adaptation to event-based systems, and the implementation of the 3D HT using Spiking Neural Networks (SNNs). Using SNN enables the proposed solution to be easily realized on hardware using FPGA, without requiring CPU or additional memory. In addition, we also discuss techniques for optimal SNN-based implementation using efficient number of neurons for the required accuracy and resolution along each dimension, without increasing the overall computational complexity. We hope that this will help to reduce the gap between event-based and frame-based systems.

show abstract

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Cited by 46 publications

References 55 publications

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

Exploiting Lightweight Statistical Learning for Event-Based Vision Processing

Hough Transform Implementation For Event-Based Systems: Concepts and Challenges

Contact Info

Product

Resources

About