A comparative analysis of tightly-coupled monocular, binocular, and stereo VINS

Paul, M. K.; Wu, Kan; Hesch, Joel A.; Nerurkar, Esha D.; Roumeliotis, Stergios I.

doi:10.1109/icra.2017.7989022

Cited by 74 publications

(54 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All three solutions are optimization-based approaches requiring powerful CPUs to operate in real time. More recently, Paul et al [27] proposed a filter-based stereo VIO based on the square root inverse filter [18], which demonstrates the possibility of operating a stereo VIO online efficiently, even on a mobile device. Since the implementations of [16] and [18] are not open-sourced, they are not used for comparison in this paper.…”

Section: Related Workmentioning

confidence: 99%

Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight

Sun

Mohta

Pfrommer

et al. 2018

IEEE Robot. Autom. Lett.

421

310

View full text Add to dashboard Cite

In recent years, vision-aided inertial odometry for state estimation has matured significantly. However, we still encounter challenges in terms of improving the computational efficiency and robustness of the underlying algorithms for applications in autonomous flight with micro aerial vehicles in which it is difficult to use high quality sensors and powerful processors because of constraints on size and weight. In this paper, we present a filter-based stereo visual inertial odometry that uses the Multi-State Constraint Kalman Filter (MSCKF) [1]. Previous work on stereo visual inertial odometry has resulted in solutions that are computationally expensive. We demonstrate that our Stereo Multi-State Constraint Kalman Filter (S-MSCKF) is comparable to state-of-art monocular solutions in terms of computational cost, while providing significantly greater robustness. We evaluate our S-MSCKF algorithm and compare it with state-of-art methods including OKVIS, ROVIO, and VINS-MONO on both the EuRoC dataset, and our own experimental datasets demonstrating fast autonomous flight with maximum speed of 17.5m/s in indoor and outdoor environments. Our implementation of the S-MSCKF is available at https://github.com/KumarRobotics/msckf vio.

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight

Sun

Mohta

Pfrommer

et al. 2018

IEEE Robot. Autom. Lett.

421

310

View full text Add to dashboard Cite

show abstract

“…A number of other approaches to visual-inertial odometry have been proposed [20], [21], [22], [23], [24], [25], [26], but these do not offer publicly-available implementations. All of these are evaluated on some of the EuRoC datasets, but none contain a comprehensive comparison to other algorithms.…”

Section: A Related Workmentioning

confidence: 99%

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Delmerico

Scaramuzza

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

357

250

View full text Add to dashboard Cite

Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. Narrated video presentation: https://youtu.be/ymI3FmwU9AY Abstract-Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. I. INTRODUCTIONVisual-inertial odometry (VIO) is currently applied to state estimation problems in a variety of domains, including autonomous vehicles, virtual and augmented reality, and flying ...

show abstract

“…While SLAM estimators -by jointly estimating the location of the sensor platform and the features in the surrounding environment -are able to easily incorporate loop closure constraints to bound localization errors and have attracted much research attention in the past three decades [8,1,6,3], there are also significant research efforts devoted to open-loop VIO systems (e.g., [30,13,14,22,17,50,37,49,53,5,2,15,40]). For example, a hybrid MSCKF/SLAM estimator was developed for VIO [23], which retains features that can be continuously tracked beyond the sliding window in the state as SLAM features while removing them when they get lost.…”

Section: Related Workmentioning

confidence: 99%

“…This localization solution has the advantages of being both cheap and ubiquitous, and has the potential to provide position and orientation (pose) estimates which are on-par in terms of accuracy with more expensive sensors such as LiDAR. To date, various algorithms are available for VINS problems including visual-inertial (VI)-SLAM [19,45] and visual-inertial odometry (VIO) [30,29,22], such as the extended Kalman filter (EKF) [30,20,14,22,17,16,50,37], unscented Kalman filter (UKF) [10,4], and batch or slidingwindow optimization methods [46,18,21,33,52,45,40], among which the EKF-based approaches remain arguably the most popular for resource constrained devices because of their efficiency. While current approaches can perform well over a short period of time in a small-scale environment (e.g., see [13,22,15]), they are not robust and accurate enough for long-term, large-scale deployments in challenging environments, due to their limited available resources of sensing, memory and computation, which, if not properly addressed, often result in short mission duration or intractable real-time estimator performance.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Schmidt-EKF for 3D Visual-Inertial SLAM

Geneva

Maley

Huang

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

It holds great implications for practical applications to enable centimeter-accuracy positioning for mobile and wearable sensor systems. In this paper, we propose a novel, high-precision, efficient visual-inertial (VI)-SLAM algorithm, termed Schmidt-EKF VI-SLAM (SEVIS), which optimally fuses IMU measurements and monocular images in a tightly-coupled manner to provide 3D motion tracking with bounded error. In particular, we adapt the Schmidt Kalman filter formulation to selectively include informative features in the state vector while treating them as nuisance parameters (or Schmidt states) once they become matured. This change in modeling allows for significant computational savings by no longer needing to constantly update the Schmidt states (or their covariance), while still allowing the EKF to correctly account for their cross-correlations with the active states. As a result, we achieve linear computational complexity in terms of map size, instead of quadratic as in the standard SLAM systems. In order to fully exploit the map information to bound navigation drifts, we advocate efficient keyframe-aided 2D-to-2D feature matching to find reliable correspondences between current 2D visual measurements and 3D map features. The proposed SEVIS is extensively validated in both simulations and experiments.

show abstract

A comparative analysis of tightly-coupled monocular, binocular, and stereo VINS

Cited by 74 publications

References 15 publications

Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight

Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

An Efficient Schmidt-EKF for 3D Visual-Inertial SLAM

Contact Info

Product

Resources

About