Activity-driven, event-based vision sensors

Delbrück, Tobi; Linares-Barranco, B.; Culurciello, Eugenio; Posch, C.

doi:10.1109/iscas.2010.5537149

Cited by 198 publications

(120 citation statements)

References 41 publications

(55 reference statements)

Supporting

Mentioning

117

Contrasting

Unclassified

Order By: Relevance

“…These last two inclusions overcome size constrains in the ADC block of Chapter 2, and output bandwidth (BW) limitations at its synchronous readout when upscaling the FPA size in high-speed applications with sparse activity in the focal plane. In this sense, a promising workaround is the use of address event representation (AER) communication protocols at pixel level [119] which, besides simplifying A/D conversion by moving part of 85 its signal processing outside the pixel, also adapt transmission capacity to visual contents themselves.…”

Section: Imager Architecture and Operation Proposalmentioning

confidence: 99%

Low-Power CMOS Digital Pixel Imagers for High-Speed Uncooled PbSe IR Applications

Margarit¹

2017

Springer Theses

View full text Add to dashboard Cite

Certifiquen ResumLa present tesi doctoral descriu la recerca i el desenvolupament d'una nova tecnologia monolítica d'imatgeria infraroja de longitud d'ona mitja (MWIR), no refrigerada i de baix cost, per a usos industrials d'alta velocitat. El treball pren el relleu dels últims avenços assolits pel soci industrial NIT S.L. en el camp dels detectors MWIR de PbSe depositats en fase vapor (VPD), afegint-hi coneixement fonamental en la investigació de noves tècniques de disseny de circuits VLSI analògics i mixtes pel desenvolupament del dispositiu integrat de lectura unit al detector pixelat.Es parteix de la hipòtesi que, mitjançant l'ús de les esmentades tècniques de disseny, les tecnologies CMOS estàndard satisfan tots els requeriments operacionals del detector VPD PbSe respecte a connectivitat, fiabilitat, funcionalitat i escalabilitat per integrar de forma econòmica el dispositiu. La càmera PbSe-CMOS resultant ha de consumir molt baixa potència, operar a freqüències de kHz, exhibir bona uniformitat, i encabir els píxels actius CMOS de lectura en el pitch compacte del pla focal de la imatge, tot atenent a les particulars característiques del detector: altes relacions de corrent d'obscuritat a senyal, elevats valors de capacitat paràsita a l'entrada i dispersions importants en el procés de fabricació.Amb la finalitat de complir amb els requisits previs, es proposen arquitectures de sensors de visió de molt baix acoblament interpíxel basades en l'ús d'una matriu de pla focal (FPA) de píxels actius exclusivament digitals. Cada píxel sensor digital (DPS) està equipat amb mòduls de comunicació d'alta velocitat, autopolarització, cancel·lació de l'offset, conversió analògica-digital (ADC) i correcció del soroll de patró fixe (FPN). El consum en cada cel·la es minimitza fent un ús exhaustiu del MOSFET operant en subllindar.L'objectiu últim és potenciar la integració de les tecnologies de sensat d'imatge infraroja (IR) basades en PbSe per expandir-ne el seu ús, no només a diferents escenaris, sinó també en diferents estadis de maduresa de la integració PbSe-CMOS. En aquest sentit, es proposa investigar un conjunt complet de blocs funcionals distribuïts en dos enfocs paral·lels: viii• Dispositius d'imatgeria MWIR "Smart" basats en frames utilitzant noves topologies de circuit DPS amb correcció de l'FPN en guany i offset. Aquesta línia de recerca exprimeix el pitch del detector per oferir una programabilitat completament digital a nivell de píxel i plena funcionalitat amb compensació de la capacitat paràsita d'entrada i memòria interna de fotograma.• Dispositius de visió MWIR "Compact"-pitch "frame-free" en base a un novedós esquema d'integració analògica en el DPS i diferenciació temporal configurable, combinats amb protocols de comunicació asíncrons dins del pla focal. Aquesta estratègia es concep per permetre una alta compactació del pitch i un increment de la velocitat de lectura, mitjançant la supressió del filtrat digital intern i l'assignació dinàmica de l'ample de banda a cada píxel de l'FPA. Aquest treball ha cond...

show abstract

Section: Imager Architecture and Operation Proposalmentioning

confidence: 99%

Low-Power CMOS Digital Pixel Imagers for High-Speed Uncooled PbSe IR Applications

Margarit¹

2017

Springer Theses

View full text Add to dashboard Cite

show abstract

“…Event-based (retinal) vision sensors [1], such as the Dynamic Vision Sensor [2], offer great potential for robotic applications: since only pixel-level brightness changes are transmitted, less bandwidth is required and less data must be processed. In addition, these changes are transmitted at the time they occur with minimal latency, which is in the order of a few micro-seconds.…”

Section: A Motivationmentioning

confidence: 99%

Lifetime estimation of events from Dynamic Vision Sensors

Mueggler

Förster

Baumli

et al. 2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We propose an algorithm to estimate the "lifetime" of events from retinal cameras, such as a Dynamic Vision Sensor (DVS). Unlike standard CMOS cameras, a DVS only transmits pixel-level brightness changes ("events") at the time they occur with micro-second resolution. Due to its low latency and sparse output, this sensor is very promising for highspeed mobile robotic applications. We develop an algorithm that augments each event with its lifetime, which is computed from the event's velocity on the image plane. The generated stream of augmented events gives a continuous representation of events in time, hence enabling the design of new algorithms that outperform those based on the accumulation of events over fixed, artificially-chosen time intervals. A direct application of this augmented stream is the construction of sharp gradient (edge-like) images at any time instant. We successfully demonstrate our method in different scenarios, including highspeed quadrotor flips, and compare it to standard visualization methods. Lifetime Estimation of Events from Dynamic Vision SensorsElias Mueggler, Christian Forster, Nathan Baumli, Guillermo Gallego and Davide Scaramuzza Abstract-We propose an algorithm to estimate the "lifetime" of events from retinal cameras, such as a Dynamic Vision Sensor (DVS). Unlike standard CMOS cameras, a DVS only transmits pixel-level brightness changes ("events") at the time they occur with micro-second resolution. Due to its low latency and sparse output, this sensor is very promising for highspeed mobile robotic applications. We develop an algorithm that augments each event with its lifetime, which is computed from the event's velocity on the image plane. The generated stream of augmented events gives a continuous representation of events in time, hence enabling the design of new algorithms that outperform those based on the accumulation of events over fixed, artificially-chosen time intervals. A direct application of this augmented stream is the construction of sharp gradient (edge-like) images at any time instant. We successfully demonstrate our method in different scenarios, including highspeed quadrotor flips, and compare it to standard visualization methods.

show abstract

“…Unlike a standard image sensor which synchronously captures a complete image in every time step, these retinas only emit spikes to indicate relevant changes in the scenery. According to a classification made by Delbruck, Linares-Barranco, Culurciello and Posch (2010), there are four types of neuromorphic retinas. Spatial contrast and spatial difference sensors avoid spatial information redundancy by only transmitting intensity ratios and differences, respectively.…”

Section: Neuromorphic Sensorsmentioning

confidence: 99%

“…For a detailed discussion of neuromorphic sensor devices, we refer the reader to the work of Liu and Delbruck (2010). A direct comparison of different types neuromorphic retina sensors is available in Delbruck et al (2010).…”

Section: Neuromorphic Sensorsmentioning

confidence: 99%

Neuromorphic implementations of neurobiological learning algorithms for spiking neural networks

2015

View full text Add to dashboard Cite

a b s t r a c tThe application of biologically inspired methods in design and control has a long tradition in robotics. Unlike previous approaches in this direction, the emerging field of neurorobotics not only mimics biological mechanisms at a relatively high level of abstraction but employs highly realistic simulations of actual biological nervous systems. Even today, carrying out these simulations efficiently at appropriate timescales is challenging. Neuromorphic chip designs specially tailored to this task therefore offer an interesting perspective for neurorobotics. Unlike Von Neumann CPUs, these chips cannot be simply programmed with a standard programming language. Like real brains, their functionality is determined by the structure of neural connectivity and synaptic efficacies. Enabling higher cognitive functions for neurorobotics consequently requires the application of neurobiological learning algorithms to adjust synaptic weights in a biologically plausible way. In this paper, we therefore investigate how to program neuromorphic chips by means of learning. First, we provide an overview over selected neuromorphic chip designs and analyze them in terms of neural computation, communication systems and software infrastructure. On the theoretical side, we review neurobiological learning techniques. Based on this overview, we then examine on-die implementations of these learning algorithms on the considered neuromorphic chips. A final discussion puts the findings of this work into context and highlights how neuromorphic hardware can potentially advance the field of autonomous robot systems. The paper thus gives an in-depth overview of neuromorphic implementations of basic mechanisms of synaptic plasticity which are required to realize advanced cognitive capabilities with spiking neural networks.

show abstract

Activity-driven, event-based vision sensors

Cited by 198 publications

References 41 publications

Low-Power CMOS Digital Pixel Imagers for High-Speed Uncooled PbSe IR Applications

Low-Power CMOS Digital Pixel Imagers for High-Speed Uncooled PbSe IR Applications

Lifetime estimation of events from Dynamic Vision Sensors

Neuromorphic implementations of neurobiological learning algorithms for spiking neural networks

Contact Info

Product

Resources

About