A 3.6 $\mu$s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor

Leñero‐Bardallo, Juan A.; Serrano-Gotarredona, Teresa; Linares-Barranco, B.

doi:10.1109/jssc.2011.2118490

Cited by 166 publications

(102 citation statements)

References 39 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…The success rate is 90% (see Table III). The third set was obtained from another type of image sensor, which is not based on differencing full frames, but on focal plane pixel light intensity temporal derivative computation and normalization with respect to ambient light [41]. When the change of light in a pixel passes a threshold, an event is triggered.…”

Section: Resultsmentioning

confidence: 99%

“…The merits of employing such an image sensor result not only from the kind of data collected, but also from the lower amount of data that needs to be communicated. The image sensor encodes the addresses of the motion-sensing pixels into a stream of events and communicates through a protocol called Address Event Representation (AER) [40][35] [41]. In AER terminology, events are communication packets that are sent from a sender to one or more receivers.…”

Section: Bio-inspired Feature Extractionmentioning

confidence: 99%

“…Several groups have proposed some version of general hardware, and the most notable is IFAT [60], from which most of our algorithmic work is inspired and targeted. Another clever and sophisticated example of general-purpose address-event processing hardware is the CAVIAR project [9], [10], [61], [41], [51], [62]. CAVIAR uses programmable convolution filters [9], [10], [51], [62] and 2D WTA and is meant for direct implementation of trained convolutional neural networks [55].…”

Section: A Hardware Implementation Considerationsmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Chen

Akselrod

et al. 2012

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-This paper proposes an algorithm for feedforward categorization of objects, and in particular human postures in realtime video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event-based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified linesegment Hausdorff-distance classifier combined with on-the-fly cluster based size and position invariant categorization. The system can achieve about 90% average successful rate in the categorization of human postures, while using only a small number of train samples. Compared to state-of-the-art bioinspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least 5 times, and is an ideal candidate for hardware implementation with event-based circuits.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Bio-inspired Feature Extractionmentioning

confidence: 99%

Section: A Hardware Implementation Considerationsmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Chen

Akselrod

et al. 2012

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Often, proposed techniques to address the ADC bottleneck involve many forms of compression, from temporal compression [17,16,11,14,4] to DCT [10] to predictive coding [15] to compressive sensing [26,24]. These new architectures require significant modifications to the system and to camera applications.…”

Section: Related Workmentioning

confidence: 99%

Energy characterization and optimization of image sensing toward continuous mobile vision

LiKamWa

Priyantha

Philipose

et al. 2013

Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services

153

View full text Add to dashboard Cite

A major hurdle to frequently performing mobile computer vision tasks is the high power consumption of image sensing. In this work, we report the first publicly known experimental and analytical characterization of CMOS image sensors. We find that modern image sensors are not energy-proportional: energy per pixel is in fact inversely proportional to frame rate and resolution of image capture, and thus image sensor systems fail to provide an important principle of energy-aware system design: trading quality for energy efficiency.We reveal two energy-proportional mechanisms, supported by current image sensors but unused by mobile systems: (i) using an optimal clock frequency reduces the power up to 50% or 30% for low-quality single frame (photo) and sequential frame (video) capturing, respectively; (ii) by entering low-power standby mode between frames, an image sensor achieves almost constant energy per pixel for video capture at low frame rates, resulting in an additional 40% power reduction. We also propose architectural modifications to the image sensor that would further improve operational efficiency. Finally, we use computer vision benchmarks to show the performance and efficiency tradeoffs that can be achieved with existing image sensors. For image registration, a key primitive for image mosaicking and depth estimation, we can achieve a 96% success rate at 3 FPS and 0.1 MP resolution. At these quality metrics, an optimal clock frequency reduces image sensor power consumption by 36% and aggressive standby mode reduces power consumption by 95%.

show abstract

“…If we compare their performance with the human retina, we can state that they perform worse in the majority of uncontrolled situations and environments: their dynamic range is much lower, their sensitivity to light is also lower, and their power consumption is much higher. For these reasons, some authors considered the idea of developing bio-inspired vision sensors (also called retinas, (Mahowald 1994, Leñero-Bardallo et al 2010, Leñero-Bardallo et al 2011. These sensors try to mimic the interactions of the cells of the human retina.…”

Section: Bio-inspired Event-based Vision Sensorsmentioning

confidence: 99%

Fire detection with a frame-less vision sensor working in the NIR band

Leñero‐Bardallo¹,

Fernández-Berni²,

Rodríguez-Vázquez³

et al. 2014

Advances in Forest Fire Research

View full text Add to dashboard Cite

This paper draws the attention of the community about the capabilities of an emerging generation of bio-inspired vision sensors to be used in fire detection systems. Their principle of operation will be described. Moreover experimental results showing the performance of an eventbased vision sensor will be provided. The sensor was intended to monitor flames activity without using optic filters. In this article, we will also extend this preliminary work and explore how its outputs can be processed to detect fire in the environment. IntroductionInfrared cameras can easily detect fire and hot spots. Unfortunately, they are expensive, difficult to handle, and fragile (Briz et al. 2003). For this reason, CMOS cameras are still preferred for some applications where it is not strictly necessary operating in the whole infrared band, (Cheon et al. 2009, Naoult et al. 2007, Bendiscio et al. 1998, Fernandez-Berni et al. 2012. Silicon can detect Near Infrared Radiation (NIR) within the band [700 1,100] nm. This property can be exploited to monitor flame and fire activity.Flames have a very characteristic oscillatory behavior. They flicker with certain frequency components that depend on their nature. The study of these frequency components is interesting for industrial processes and applications. Traditionally CMOS cameras with NIR filters have been employed (Yan et al. 2006). As an alternative to this traditional approach, recently we proposed an bio-inspired event-based system to monitor flame activity without using optic filters (Leñero-Bardallo et al. 2013). The idea behind this method was to compare the photocurrents of stacked photodiodes at different depths. The top one was more sensitive to shorter wavelengths and the bottom one was more sensitive to longer wavelengths, having a very good sensitivity within the NIR band. Just comparing the difference between their photocurrents, we were able to determine if the incident radiation was within the NIR band without optic filters. Moreover, we could process with a real-time algorithm the temporal variations of the NIR levels in the visual scene provoked by flames and compute their frequency components values. This paper studies how the prior work to monitor flame activity could be extended to detect fire in the environment and emit an alarm in case. We will show that the previously implemented algorithm outputs can be processed to detect fire with a low computational load. Up

show abstract

A 3.6 $\mu$s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor

Cited by 166 publications

References 39 publications

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

Energy characterization and optimization of image sensing toward continuous mobile vision

Fire detection with a frame-less vision sensor working in the NIR band

Contact Info

Product

Resources

About