Efficient Visual-Inertial Navigation using a Rolling-Shutter Camera with Inaccurate Timestamps

Guo, Chao; Kottas, Dimitrios G.; DuToit, Ryan C.; Ahmed, Ahmed; Li, Ruipeng; Roumeliotis, Stergios I.

doi:10.15607/rss.2014.x.057

Cited by 43 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our algorithm to what is considered to be the state-of-the-art VINS in terms of accuracy and speed on mobile devices, the MSC-KF 6 [20,10,8], as well as our efficient single-precision implementation of the shortterm smoother (STS) of [4], which is a square-root inverse sliding window filter that does not take advantage of the problem structure. For fair comparisons, all three algorithms use the same feature selection scheme and estimate the timesynchronization, rolling shutter, and extrinsic parameters.…”

Section: Resultsmentioning

confidence: 99%

“…where Ii q G is the quaternion representation of the orientation of the global frame {G} in the IMU's frame of reference {I i }, G p Ii is the position of {I i } in {G}, and t di is the IMU-camera time offset, 3 at time step i, similar to the definition in [8]. The parameter state vector, x P , consists of the constant parameters:…”

Section: A System Statementioning

confidence: 99%

“…In addition, there exists a time-varying offset between the IMU and the camera's time stamps (i.e., the two clocks are not synchronized). To handle these issues, we use the interpolation model of [8], where each camera pose is interpolated using the two closest cloned poses. In particular, the measurement model is:…”

Section: Visual Measurement Equations and Corresponding Cost Termsmentioning

confidence: 99%

“…That, as well as the results in Table I lead us to believe that all three algorithms achieve comparable levels of accuracy. 8 …”

Section: A Localization Accuracymentioning

confidence: 99%

“…Popular filtering techniques for VINS are either based on the extended Kalman filter (EKF) (e.g., [20,10,8]) or the inverse filter (INVF) (e.g., [23,6,15]). While real-time performance on mobile devices has been successfully demonstrated using EKFs (e.g., [17,8]), to the best of our knowledge, no real-time INVF-based system has been shown on resource-constrained platforms such as cell phones.…”

Section: Introduction and Related Workmentioning

confidence: 99%

See 4 more Smart Citations

A Square Root Inverse Filter for Efficient Vision-aided Inertial Navigation on Mobile Devices

Wu¹,

Ahmed

Γεωργίου

et al. 2015

Robotics: Science and Systems XI

Self Cite

109

View full text Add to dashboard Cite

Abstract-In this paper, we present a square-root inverse sliding window filter (SR-ISWF) for vision-aided inertial navigation systems (VINS). While regular inverse filters suffer from numerical issues, employing their square-root equivalent enables the usage of single-precision number representations, thus achieving considerable speed ups as compared to doubleprecision alternatives on resource-constrained mobile platforms. Besides a detailed description of the SR-ISWF for VINS, which focuses on the numerical procedures that enable exploiting the problem's structure for gaining in efficiency, this paper presents a thorough validation of the algorithm's processing requirements and achieved accuracy. In particular, experiments are conducted using a commercial-grade cell phone, where the proposed algorithm is shown to achieve the same level of estimation accuracy, when compared to state-of-the-art VINS algorithms, with significantly higher speed.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: A System Statementioning

confidence: 99%

Section: Visual Measurement Equations and Corresponding Cost Termsmentioning

confidence: 99%

“…That, as well as the results in Table I lead us to believe that all three algorithms achieve comparable levels of accuracy. 8 …”

Section: A Localization Accuracymentioning

confidence: 99%

Section: Introduction and Related Workmentioning

confidence: 99%

See 3 more Smart Citations

A Square Root Inverse Filter for Efficient Vision-aided Inertial Navigation on Mobile Devices

Wu¹,

Ahmed

Γεωργίου

et al. 2015

Robotics: Science and Systems XI

Self Cite

109

View full text Add to dashboard Cite

show abstract

Laser–visual–inertial odometry and mapping with high robustness and low drift

Zhang¹,

Singh²

2018

Journal of Field Robotics

197

115

View full text Add to dashboard Cite

We present a data processing pipeline to online estimate ego-motion and build a map of the traversed environment, leveraging data from a 3D laser scanner, a camera, and an inertial measurement unit (IMU). Different from traditional methods that use a Kalman filter or factor-graph optimization, the proposed method employs a sequential, multilayer processing pipeline, solving for motion from coarse to fine.Starting with IMU mechanization for motion prediction, a visual-inertial coupled method estimates motion; then, a scan matching method further refines the motion estimates and registers maps. The resulting system enables high-frequency, lowlatency ego-motion estimation, along with dense, accurate 3D map registration.Further, the method is capable of handling sensor degradation by automatic reconfiguration bypassing failure modules. Therefore, it can operate in the presence of highly dynamic motion as well as in the dark, texture-less, and structure-less environments. During experiments, the method demonstrates 0.22% of relative position drift over 9.3 km of navigation and robustness w.r.t. running, jumping, and even highway speed driving (up to 33 m/s).mapping, ego-motion estimation, 3D laser scanner, vision, inertial sensor 1 | INTRODUCTION This paper aims at developing a method for online ego-motion estimation with data from a 3D laser scanner, a camera, and an inertial measurement unit (IMU). The estimated motion further registers laser points to build a map of the traversed environment. In many real-world applications, ego-motion estimation and mapping must be conducted in real time. In an autonomous navigation system, the map is crucial for motion planning and obstacle avoidance, while the motion estimation is important for vehicle control and maneuver.Very often, high-accuracy global positioning system (GPS)-inertial navigation system (INS) solutions are impractical when the application is GPS-denied, lightweight, or cost-sensitive. The proposed method utilizes only perception sensors without reliance on GPS.Specially, we are interested in solving for extremely aggressive motion. Yet, it remains nonobvious how to solve the problem reliably in 6 degrees of freedom (DOF), in real time, and in a small form factor. The problem is closely relevant to sensor degradation due to sparsity of the data during dynamic maneuver. To the best of our knowledge, the proposed method is by far the first to enable such high-rate ego-motion estimation capable of handling running and jumping (see Figure 1 as an example), while at the same time develop a dense, accurate 3D map, in the field under various lighting and structural conditions, and using only sensing and computing devices that can be easily carried by a person.The key reason that enables this level of performance is our novel way of data processing. As shown in Figure 2a, a standard Kalman filter-based method typically uses IMU mechanization in a prediction step followed by update steps seeded with individual visual features or laser landmarks. On the other hand, a fa...

show abstract

Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry

Ling

Bao

Jie

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Combining cameras and inertial measurement units (IMUs) has been proven effective in motion tracking, as these two sensing modalities offer complementary characteristics that are suitable for fusion. While most works focus on global-shutter cameras and synchronized sensor measurements, consumer-grade devices are mostly equipped with rolling-shutter cameras and suffer from imperfect sensor synchronization. In this work, we propose a nonlinear optimization-based monocular visual inertial odometry (VIO) with varying camera-IMU time offset modeled as an unknown variable. Our approach is able to handle the rollingshutter effects and imperfect sensor synchronization in a unified way. Additionally, we introduce an efficient algorithm based on dynamic programming and red-black tree to speed up IMU integration over variablelength time intervals during the optimization. An uncertainty-aware initialization is also presented to launch the VIO robustly. Comparisons with state-of-the-art methods on the Euroc dataset and mobile phone data are shown to validate the effectiveness of our approach.

show abstract

Efficient Visual-Inertial Navigation using a Rolling-Shutter Camera with Inaccurate Timestamps

Cited by 43 publications

References 15 publications

A Square Root Inverse Filter for Efficient Vision-aided Inertial Navigation on Mobile Devices

A Square Root Inverse Filter for Efficient Vision-aided Inertial Navigation on Mobile Devices

Laser–visual–inertial odometry and mapping with high robustness and low drift

Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry

Contact Info

Product

Resources

About