Event-Based Visual Inertial Odometry

Zhu, Alex Zihao; Atanasov, Nikolay; Daniilidis, Kostas

doi:10.1109/cvpr.2017.616

Cited by 169 publications

(146 citation statements)

References 73 publications

Supporting

Mentioning

145

Contrasting

Unclassified

Order By: Relevance

“…The original MSCKF algorithm in [1] proposed a measurement model that expressed the geometric constraints between all of the camera poses that observed a particular image feature, without the need to maintain an estimate of the 3D feature position in the state. The extended Kalman filter backend in [29] implements this formulation of the MSCKF for event-based camera inputs, but has been adapted to feature tracks from standard cameras. At the time of publication of this paper, this MSCKF implementation will be publicly available.…”

Section: A Msckfmentioning

confidence: 99%

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Delmerico

Scaramuzza

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

356

250

View full text Add to dashboard Cite

Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. Narrated video presentation: https://youtu.be/ymI3FmwU9AY Abstract-Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. I. INTRODUCTIONVisual-inertial odometry (VIO) is currently applied to state estimation problems in a variety of domains, including autonomous vehicles, virtual and augmented reality, and flying ...

show abstract

Section: A Msckfmentioning

confidence: 99%

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Delmerico

Scaramuzza

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

356

250

View full text Add to dashboard Cite

show abstract

“…In recent years many researches has focused on the fusion of multiple sensors to achieve higher accuracy and robustness. In [14] and [15], the authors combined the event camera with IMU. In [16], Vidal et al even combined three kind of sensors: the event camera, IMU and the grayscale camera, to provide a mature engineering application.…”

Section: Related Workmentioning

confidence: 99%

“…The main contribution of this paper is to propose a event based in-vehicle visual odometry system that utilizing the feature tracking algorithm proposed by Alex Zhu et al in [14]. The framework of our visual odometry system is shown in Fig.1.…”

Section: Trackingmentioning

confidence: 99%

Neuromorphic Visual Odometry System For Intelligent Vehicle Application With Bio-inspired Vision Sensor

Zhu

Dong

et al. 2019

2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)

View full text Add to dashboard Cite

The neuromorphic camera is a brand new vision sensor that has emerged in recent years. In contrast to the conventional frame-based camera, the neuromorphic camera only transmits local pixel-level changes at the time of its occurrence and provides an asynchronous event stream with low latency. It has the advantages of extremely low signal delay, low transmission bandwidth requirements, rich information of edges, high dynamic range etc., which make it a promising sensor in the application of in-vehicle visual odometry system. This paper proposes a neuromorphic in-vehicle visual odometry system using feature tracking algorithm. To the best of our knowledge, this is the first in-vehicle visual odometry system that only uses a neuromorphic camera, and its performance test is carried out on actual driving datasets. In addition, an in-depth analysis of the results of the experiment is provided. The work of this paper verifies the feasibility of in-vehicle visual odometry system using neuromorphic cameras.

show abstract

“…(Bottom row) IMU measurements recording the trajectory of the camera motion. (a) Ground Truth (b) DAViS-OF[46] (c) LK-DVS[32] (d) EV-FlowNet[53] (e) DistSurf-OF Optical flow results on DVSMOTION20 sequences (shown with denoising). Arrow orientation and magnitude indicate the estimated pixel motion orientation and speed of the observed events.…”

mentioning

confidence: 99%

Distance Surface for Event-Based Optical Flow

Almatrafi

Baldwin

Aizawa

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We propose DistSurf-OF, a novel optical flow method for neuromorphic cameras. Neuromorphic cameras (or event detection cameras) are an emerging sensor modality that makes use of dynamic vision sensors (DVS) to report asynchronously the log-intensity changes (called "events") exceeding a predefined threshold at each pixel. In absence of the intensity value at each pixel location, we introduce a notion of "distance surface"-the distance transform computed from the detected events-as a proxy for object texture. The distance surface is then used as an input to the intensity-based optical flow methods to recover the two dimensional pixel motion. Real sensor experiments verify that the proposed DistSurf-OF accurately estimates the angle and speed of each events.

show abstract

Event-Based Visual Inertial Odometry

Cited by 169 publications

References 73 publications

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Neuromorphic Visual Odometry System For Intelligent Vehicle Application With Bio-inspired Vision Sensor

Distance Surface for Event-Based Optical Flow

Contact Info

Product

Resources

About