Multi‐Event‐Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion

Abstract

Event cameras are bio‐inspired sensors that offer advantages over traditional cameras. They operate asynchronously, sampling the scene at microsecond resolution and producing a stream of brightness changes. This unconventional output has sparked novel computer vision methods to unlock the camera's potential. Here, the problem of event‐based stereo 3D reconstruction for SLAM is considered. Most event‐based stereo methods attempt to exploit the high temporal resolution of the camera and the simultaneity of events across cameras to establish matches and estimate depth. By contrast, this work investigates how to estimate depth without explicit data association by fusing disparity space images (DSIs) originated in efficient monocular methods. Fusion theory is developed and applied to design multi‐camera 3D reconstruction algorithms that produce state‐of‐the‐art results, as confirmed by comparisons with four baseline methods and tests on a variety of available datasets.

Event-based Stereo Depth Estimation from Ego-motion using Ray Density Fusion

“…Having one DSI per camera allows us to preserve the origin of the event data, and having geometrically aligned DSIs avoids resampling errors during fusion. We then fuse the DSIs using element-wise operations like the Generalized means, which satisfy an order relationship [8]:…”

“…While [8] evaluates Alg. 1 on a variety of datasets, here we present depth estimation results on the egocentric TUM-VIE dataset [10] for the sake of brevity.…”

“…Moreover, the outstanding properties of event cameras, such as high dynamic range (HDR), high temporal resolution (≈ µs) and low power consumption, offer potential to tackle scenarios that are challenging for standard cameras (high speed and/or HDR) [1,3,6,7,15,16,18]. This extended abstract is based on our recent paper on multi-event camera depth estimation [8]. Similar to how humans exploit ego-motion of their eyes to explore and perceive the 3D scene, we use the motion of stereo event cameras to estimate scene depth (i.e., 3D reconstruction).…”

“…Using Space Sweeping, it builds ray density Disparity Space Images (DSIs) from each camera data and fuses them into a single DSI, from which depth is recovered. Image adapted with permission from [8].…”

“…In contrast, in [8] we investigate a novel stereo approach, without explicitly establishing event matches (e.g., via event coincidence). Correspondencefree depth estimation for monocular event cameras has been shown to generate state-of-the-art results in visual odometry [17].…”