Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry

Wisth, David; Camurri, Marco; Das, Sandipan; Fallon, Maurice

doi:10.1109/lra.2021.3056380

Cited by 88 publications

(49 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zuo [14] introduced LIC-Fusion which tightly coupled lidar edge features, spares visual features, as well as plane features together by using the MSCKF framework [15]. David [16] presents a unified multi-sensor odometry that jointly optimizes 3D primitives such as lines and planes. However, since the measurements of these methods are highly tightly coupled, they are difficult to extend to other sensors.…”

Section: B Tightly Coupled Laser-visual-inertial Odometrymentioning

confidence: 99%

Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments

Zhao¹,

Zhang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose Super Odometry, a high-precision multi-modal sensor fusion framework, providing a simple but effective way to fuse multiple sensors such as LiDAR, camera, and IMU sensors and achieve robust state estimation in perceptually-degraded environments. Different from traditional sensor-fusion methods, Super Odometry employs an IMUcentric data processing pipeline, which combines the advantages of loosely coupled methods with tightly coupled methods and recovers motion in a coarse-to-fine manner. The proposed framework is composed of three parts: IMU odometry, visualinertial odometry, and laser-inertial odometry. The visualinertial odometry and laser-inertial odometry provide the pose prior to constrain the IMU bias and receive the motion prediction from IMU odometry. To ensure high performance in real-time, we apply a dynamic octree that only consumes 10% of the running time compared with a static KD-tree. The proposed system was deployed on drones and ground robots, as part of Team Explorer's effort to the DARPA Subterranean Challenge where the team won 1 st and 2 nd place in the Tunnel and Urban Circuits 1 , respectively.

show abstract

Section: B Tightly Coupled Laser-visual-inertial Odometrymentioning

confidence: 99%

Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments

Zhao¹,

Zhang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Wisth et al [10] used factor graphs to fuse visual information with inertial and kinematics. In [11], the same authors improved on the factor graph formulation by adding feature tracking-based lidar measurements.…”

Section: Kinematic-inertial State Estimation In Legged Robotsmentioning

confidence: 99%

Learning Inertial Odometry for Dynamic Legged Robot State Estimation

Buchanan¹,

Camurri²,

Dellaert³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper introduces a novel proprioceptive state estimator for legged robots based on a learned displacement measurement from IMU data. Recent research in pedestrian tracking has shown that motion can be inferred from inertial data using convolutional neural networks. A learned inertial displacement measurement can improve state estimation in challenging scenarios where leg odometry is unreliable, such as slipping and compressible terrains. Our work learns to estimate a displacement measurement from IMU data which is then fused with traditional leg odometry. Our approach greatly reduces the drift of proprioceptive state estimation, which is critical for legged robots deployed in vision and lidar denied environments such as foggy sewers or dusty mines. We compared results from an EKF and an incremental fixed-lag factor graph estimator using data from several real robot experiments crossing challenging terrains. Our results show a reduction of relative pose error by 37 % in challenging scenarios when compared to a traditional kinematic-inertial estimator without learned measurement. We also demonstrate a 22 % reduction in error when used with vision systems in visually degraded environments such as an underground mine.

show abstract

“…This bounds computational time and achieves the same accuracy compared to using all available features; • Extensive experimental evaluation across a range of scenarios demonstrating superior robustness, particularly when VIO with an individual camera fails. The proposed algorithm, VILENS-MC, builds upon our previous VILENS estimation system [7], [8], by fusing multiple cameras and improving front-end feature processing.…”

Section: A Contributionmentioning

confidence: 99%

“…To generate ground truth, ICP was used to align the current LIDAR scan to detailed prior maps, collected using a commercial laser mapping system. The high frequency motion estimate from the IMU was used to carefully remove lidar motion distortion [7]. For an in-depth discussion on ground truth generation the reader is referred to [29].…”

Section: A Datasetsmentioning

confidence: 99%

Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry

Zhang,

Wisth,

Camurri

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a multi-camera visual-inertial odometry system based on factor graph optimization which estimates motion by using all cameras simultaneously while retaining a fixed overall feature budget. We focus on motion tracking in challenging environments such as in narrow corridors and dark spaces with aggressive motions and abrupt lighting changes. These scenarios cause traditional monocular or stereo odometry to fail. While tracking motion across extra cameras should theoretically prevent failures, it causes additional complexity and computational burden. To overcome these challenges, we introduce two novel methods to improve multi-camera feature tracking. First, instead of tracking features separately in each camera, we track features continuously as they move from one camera to another. This increases accuracy and achieves a more compact factor graph representation. Second, we select a fixed budget of tracked features which are spread across the cameras to ensure that the limited computational budget is never exceeded. We have found that using a smaller set of informative features can maintain the same tracking accuracy while reducing backend optimization time. Our proposed method was extensively tested using a hardware-synchronized device containing an IMU and four cameras (a front stereo pair and two lateral) in scenarios including an underground mine, large open spaces, and building interiors with narrow stairs and corridors. Compared to stereoonly state-of-the-art VIO methods, our approach reduces the drift rate (RPE) by up to 80 % in translation and 39 % in rotation.

show abstract

Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry

Cited by 88 publications

References 28 publications

Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments

Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments

Learning Inertial Odometry for Dynamic Legged Robot State Estimation

Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry

Contact Info

Product

Resources

About