Asynchronous Event-Based Binocular Stereo Matching

Rogister, P.; Benosman, Ryad; Ieng, Sio-Hoï; Lichtsteiner, P.; Delbrück, Tobi

doi:10.1109/tnnls.2011.2180025

Cited by 139 publications

(137 citation statements)

References 19 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…Event cameras have been used for object tracking [Delbruck and Lichtsteiner, 2007, Drazen et al, 2011, Delbruck and Lang, 2013, surveillance and monitoring [Litzenberger et al, 2006, Piatkowska et al, 2012, object recognition [Wiesmann et al, 2012, Orchard et al, 2015, Lagorce et al, 2016 and gesture control [Lee et al, 2014]. They have also been used for stereo depth estimation [Rogister et al, 2012, Piatkowska et al, 2013 (see also related work in Section 3), 3D panoramic imaging [Schraml et al, 2015], structured light 3D scanning Fig. 2: Comparison of the output of a standard camera and an event camera (DVS) when viewing a spinning disk with a black circle.…”

Section: Event Cameras and Applicationsmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of eventbased multi-view stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (1) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any pre-processing operationand (2) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semi-dense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. Abstract Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Event-based Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semidense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semi-dense geometric information without any preprocessing operation-and (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps, without requiring any explicit data association or intensity estimation. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU.

show abstract

Section: Event Cameras and Applicationsmentioning

confidence: 99%

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

et al. 2017

View full text Add to dashboard Cite

show abstract

“…These methods follow a two-step approach: first they solve the event correspondence problem across image planes and then triangulate the location of the 3D point. Events are matched in two ways: either using traditional stereo methods on artificial frames generated by accumulating events over time [7,11], or exploiting simultaneity and temporal correlations of the events across sensors [2,6,8,10].…”

Section: Related Work On Event-based Depth Estimationmentioning

confidence: 99%

EMVS: Event-based Multi-View Stereo

Rebecq¹,

Gallego²,

Scaramuzza³

2016

Procedings of the British Machine Vision Conference 2016

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Eventbased Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semidense geometric information without any pre-processing operationand (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. Abstract Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the output is composed of a sequence of asynchronous events rather than actual intensity images, traditional vision algorithms cannot be applied, so that a paradigm shift is needed. We introduce the problem of Event-based Multi-View Stereo (EMVS) for event cameras and propose a solution to it. Unlike traditional MVS methods, which address the problem of estimating dense 3D structure from a set of known viewpoints, EMVS estimates semi-dense 3D structure from an event camera with known trajectory. Our EMVS solution elegantly exploits two inherent properties of an event camera: (i) its ability to respond to scene edges-which naturally provide semidense geometric information without any pre-processing operation-and (ii) the fact that it provides continuous measurements as the sensor moves. Despite its simplicity (it can be implemented in a few lines of code), our algorithm is able to produce accurate, semidense depth maps. We successfully validate our method on both synthetic and real data. Our method is computationally very efficient and runs in real-time on a CPU. REBECQ, GALLEGO, SCARAMUZZA: EMVS: EVENT-BASED MULTI-VIEW STEREO

show abstract

“…One approach to acquire depth information is a fully event-based stereo vision set-up consisting of two eDVS sensors. While such a solution has been demonstrated [10], the capabilities are currently rather limited and not yet sufficient for deploying a 3D SLAM method. In this work we chose to provide depth information by a dedicated depth sensor like the PrimeSense RGB-D sensor.…”

Section: The D-edvs Sensormentioning

confidence: 99%

Event-based 3D SLAM with a depth-augmented dynamic vision sensor

Weikersdorfer

Adrian

Cremers

et al. 2014

2014 IEEE International Conference on Robotics and Automation (ICRA)

115

View full text Add to dashboard Cite

Abstract-We present the D-eDVS-a combined event-based 3D sensor -and a novel event-based full-3D simultaneous localization and mapping algorithm which works exclusively with the sparse stream of visual data provided by the D-eDVS. The D-eDVS is a combination of the established PrimeSense RGB-D sensor and a biologically inspired embedded dynamic vision sensor. Dynamic vision sensors only react to dynamic contrast changes and output data in form of a sparse stream of events which represent individual pixel locations. We demonstrate how an event-based dynamic vision sensor can be fused with a classic frame-based RGB-D sensor to produce a sparse stream of depth-augmented 3D points. The advantages of a sparse, event-based stream are a much smaller amount of generated data, thus more efficient resource usage, and a continuous representation of motion allowing lag-free tracking. Our eventbased SLAM algorithm is highly efficient and runs 20 times faster than realtime, provides localization updates at several hundred Hertz, and produces excellent results. We compare our method against ground truth from an external tracking system and two state-of-the-art algorithms on a new dataset which we release in combination with this paper.

show abstract

Asynchronous Event-Based Binocular Stereo Matching

Cited by 139 publications

References 19 publications

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time

EMVS: Event-based Multi-View Stereo

Event-based 3D SLAM with a depth-augmented dynamic vision sensor

Contact Info

Product

Resources

About