VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator

Qin, Tong; Li, Peiliang; Shen, Shaojie

doi:10.1109/tro.2018.2853729

Cited by 2,669 publications

(1,826 citation statements)

References 45 publications

Supporting

Mentioning

1,806

Contrasting

Unclassified

Order By: Relevance

“…Such representations can be sparse [184], semidense [185], or fully dense [186]. While dense representations can be directly used for autonomous navigation [186] or geographical reference, sparse representations are often only used for state estimation [187] or collaborative control of robotic agents. Due to the fact that the environment is often only partially known or even totally unknown, mapping tasks are often coupled with localization (pose estimation) issues, which turn them into the classic simultaneous localization and mapping (SLAM) problem.…”

Section: Cooperative Aerial Mappingmentioning

confidence: 99%

A Survey on Aerial Swarm Robotics

et al. 2018

Self Cite

View full text Add to dashboard Cite

Abstract-The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional (3-D) space, and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of the paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas.

show abstract

Section: Cooperative Aerial Mappingmentioning

confidence: 99%

A Survey on Aerial Swarm Robotics

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among the methods evaluated in this paper, the recently proposed VINS-Mono pipeline [4] compares to OKVIS, but only on a few of the EuRoC datasets. In [2], OKVIS was compared to a non-public implementation of MSCKF [1] on non-public datasets.…”

Section: A Related Workmentioning

confidence: 99%

“…The software is available in both a ROScompatible PC version and an iOS implementation for state estimation on mobile devices. 4 …”

Section: Roviomentioning

confidence: 99%

See 1 more Smart Citation

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Delmerico

Scaramuzza

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

340

250

View full text Add to dashboard Cite

Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. Narrated video presentation: https://youtu.be/ymI3FmwU9AY Abstract-Flying robots require a combination of accuracy and low latency in their state estimation in order to achieve stable and robust flight. However, due to the power and payload constraints of aerial platforms, state estimation algorithms must provide these qualities under the computational constraints of embedded hardware. Cameras and inertial measurement units (IMUs) satisfy these power and payload constraints, so visualinertial odometry (VIO) algorithms are popular choices for state estimation in these scenarios, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is not clear from existing results in the literature, however, which VIO algorithms perform well under the accuracy, latency, and computational constraints of a flying robot with onboard state estimation. This paper evaluates an array of publicly-available VIO pipelines (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, and SVO+GTSAM) on different hardware configurations, including several singleboard computer systems that are typically found on flying robots. The evaluation considers the pose estimation accuracy, per-frame processing time, and CPU and memory load while processing the EuRoC datasets, which contain six degree of freedom (6DoF) trajectories typical of flying robots. We present our complete results as a benchmark for the research community. I. INTRODUCTIONVisual-inertial odometry (VIO) is currently applied to state estimation problems in a variety of domains, including autonomous vehicles, virtual and augmented reality, and flying ...

show abstract

“…The quadrotor is equipped with dual fisheye cameras with one upward facing and another downward facing. The localization is carried out by visual-inertial fusion (Qin, Li, & Shen, 2018) in the onboard CPU, and the depths of each pixel in images are estimated using stereo matching in an additional graphics processing unit (GPU). Compared with the above-mentioned LiDAR-based mapping system, the pixelwise depth estimation by the fisheye cameras is significantly poorer and has a nonnegligible latency.…”

Section: Test With a Degraded Sensormentioning

confidence: 99%

Flying on point clouds: Online trajectory generation and autonomous navigation for quadrotors in cluttered environments

Gao

et al. 2018

Journal of Field Robotics

Self Cite

132

View full text Add to dashboard Cite

Micro aerial vehicles (MAVs), especially quadrotors, have been widely used in field applications, such as disaster response, field surveillance, and search‐and‐rescue. For accomplishing such missions in challenging environments, the capability of navigating with full autonomy while avoiding unexpected obstacles is the most crucial requirement. In this paper, we present a framework for online generating safe and dynamically feasible trajectories directly on the point cloud, which is the lowest‐level representation of range measurements and is applicable to different sensor types. We develop a quadrotor platform equipped with a three‐dimensional (3D) light detection and ranging (LiDAR) and an inertial measurement unit (IMU) for simultaneously estimating states of the vehicle and building point cloud maps of the environment. Based on the incrementally registered point clouds, we online generate and refine a flight corridor, which represents the free space that the trajectory of the quadrotor should lie in. We represent the trajectory as piecewise Bézier curves by using the Bernstein polynomial basis and formulate the trajectory generation problem as a convex program. By using Bézier curves, we can constrain the position and kinodynamics of the trajectory entirely within the flight corridor and given physical limits. The proposed approach is implemented to run onboard in real‐time and is integrated into an autonomous quadrotor platform. We demonstrate fully autonomous quadrotor flights in unknown, complex environments to validate the proposed method.

show abstract

VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator

Cited by 2,669 publications

References 45 publications

A Survey on Aerial Swarm Robotics

A Survey on Aerial Swarm Robotics

A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots

Flying on point clouds: Online trajectory generation and autonomous navigation for quadrotors in cluttered environments

Contact Info

Product

Resources

About