Fun with Asynchronous Vision Sensors and Processing

Abstract

Abstract. This paper provides a personal perspective on our group's efforts in building event-based vision sensors, algorithms, and applications over the period [2002][2003][2004][2005][2006][2007][2008][2009][2010][2011][2012]. Some recent advances from other groups are also briefly described.When Mahowald and Mead built the first silicon retina with asynchronous digital output around 1992 [1], conventional CMOS active pixel sensors (APS) were still research chips. It required the investment by industry of about a billion dollars to bring CMOS APS to high volume production. So it is no surprise that while the imager community has been consumed by the megapixel race to make nice photos, cameras that mimic more closely how the eye works have taken a long time to come to a useful form. These "silicon retinas" are much more complex at the pixel level than APS cameras and they pay the price in terms of fill factor and pixel size; machine vision cameras with capability of synchronous global electronic shutter are about 5um. Silicon retina pixels are roughly 10 times the area of a machine vision camera pixel. So why are silicon retinas still interesting? Mostly because of the high cost at the system level of processing the highly redundant data from conventional cameras, and the fixed latencies imposed by the frame intervals. High performance activity driven event-based sensors could greatly benefit applications in real time robotics, where just as in nature, latency and power are very important [2,5,9,10].

Evaluation of Event-Based Algorithms for Optical Flow with Ground-Truth from Inertial Measurement Sensor

“…Readers are referred to the source code for implementation details. DS was inspired by the organization of V1 direction selective simple cells (Delbruck, 2008 , 2012 ).…”

Event-Based Vision: A Survey

“…The DVS event camera [2] had its genesis in a frame-based silicon retina design where the continuous-time photoreceptor was capacitively coupled to a readout circuit that was reset each time the pixel was sampled [44]. More recent event camera technology has been reviewed in the electronics and neuroscience literature [10], [36], [38], [45], [46], [47]. Although surprisingly many applications can be solved by only processing DVS events (i.e., brightness changes), it became clear that some also require some form of static output (i.e., "absolute" brightness).…”

Bio-Inspired Optic Flow from Event-Based Neuromorphic Sensor Input

“…For example, in an assisted living scenario a bank of such sensors can successfully be utilized to detect and classify fast vertical motions as unintended occasional falls, indicative of an preemptive alarm situation [Fu et al, 2008]. Tracking scenarios have been investigated to steer real-world interaction [Delbrück, 2012] as well as in microbiology set-ups [Drazen et al, 2011]. Basic research investigations have been reported as well.…”