Rolling-Shutter Modelling for Direct Visual-Inertial Odometry

Schubert, David; Demmel, Nikolaus; Stumberg, Lukas von; Usenko, Vladyslav; Cremers, Daniel

doi:10.1109/iros40897.2019.8968539

Cited by 24 publications

(24 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, modeling the RS effect leads to better performance with RS cameras. This improvement has been validated for video stabilization [24], structure from motion [6,25,18,4], dense mapping [11], vision-aided odometry [14,20,26].…”

Section: Related Workmentioning

confidence: 96%

Continuous-Time Spatiotemporal Calibration of a Rolling Shutter Camera-IMU System

Huai¹,

Zhuang²,

Yuan³

et al. 2021

Preprint

View full text Add to dashboard Cite

The rolling shutter (RS) mechanism is widely used by consumer-grade cameras, which are essential parts in smartphones and autonomous vehicles. The RS effect leads to image distortion upon relative motion between a camera and the scene. This effect needs to be considered in video stabilization, structure from motion, and vision-aided odometry, for which recent studies have improved earlier global shutter (GS) methods by accounting for the RS effect. However, it is still unclear how the RS affects spatiotemporal calibration of the camera in a sensor assembly, which is crucial to good performance in aforementioned applications.This work takes the camera-IMU system as an example and looks into the RS effect on its spatiotemporal calibration. To this end, we develop a calibration method for a RScamera-IMU system with continuous-time B-splines by using a calibration target. Unlike in calibrating GS cameras, every observation of a landmark on the target has a unique camera pose fitted by continuous-time B-splines. With simulated data generated from four sets of public calibration data, we show that RS can noticeably affect the extrinsic parameters, causing errors about 1 • in orientation and 2 cm in translation with a RS setting as in common smartphone cameras. With real data collected by two industrial camera-IMU systems, we find that considering the RS effect gives more accurate and consistent spatiotemporal calibration. Moreover, our method also accurately calibrates the inter-line delay of the RS. The code for simulation and calibration is publicly available 1 .

show abstract

Section: Related Workmentioning

confidence: 96%

Continuous-Time Spatiotemporal Calibration of a Rolling Shutter Camera-IMU System

Huai¹,

Zhuang²,

Yuan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They jointly optimize the camera poses, 3D scene structure as well as camera intrinsic parameters by maximizing the photometric consistency. The representative works are LSD-SLAM [8], DSO [7] and their many variants [29,20,19,10]. Different from direct method, feature-based methods extract a set of sparse keypoints from the raw images which are then matched across different views.…”

Section: Related Workmentioning

confidence: 99%

MBA-VO: Motion Blur Aware Visual Odometry

Liu¹,

Zuo²,

Larsson³

et al. 2021

Preprint

View full text Add to dashboard Cite

Motion blur is one of the major challenges remaining for visual odometry methods. In low-light conditions where longer exposure times are necessary, motion blur can appear even for relatively slow camera motions. In this paper we present a novel hybrid visual odometry pipeline with direct approach that explicitly models and estimates the camera's local trajectory within the exposure time. This allows us to actively compensate for any motion blur that occurs due to the camera motion. In addition, we also contribute a novel benchmarking dataset for motion blur aware visual odometry. In experiments we show that by directly modeling the image formation process, we are able to improve robustness of the visual odometry, while keeping comparable accuracy as that for images without motion blur.

show abstract

“…The synthetic sequences are extended from the ICL-NUIM dataset, the real data sequences are recorded along with the ground truth trajectory from a motion capture system. Schubert et al [14] provide a real dataset that contains GS and RS images, IMU data and ground truth pose for ten different sequences.…”

Section: Introductionmentioning

confidence: 99%

“…All sequences provide accurate ground truth, thus researchers can accurately know the error of a specific method. Different from Kerl et al [13] the dataset of this paper contains IMU data, different from Schubert et al [14], the dataset in this paper is synthetic and each trajectory contains three sequences of different motion speeds (i. e., slow, medium and fast corresponding to different rolling shutter effects), and the IMU data in this dataset are recorded at various rates from 90Hz to 14400 Hz.…”

Section: Introductionmentioning

confidence: 99%

The WHU Rolling Shutter Visual-Inertial Dataset

2020

View full text Add to dashboard Cite

The vast majority of modern consumer cameras employ a rolling shutter (RS) mechanism which has a price and electronic advantage to global shutter (GS). However, in geometric computer vision applications such as visual simultaneous localization and mapping (VSLAM), performances of accuracy and robustness are usually deteriorated due to the rolling shutter effect when using the RS cameras. This paper introduced the Wuhan University Rolling Shutter Visual-Inertial (WHU-RSVI) synthetic dataset for evaluating VSLAM and VI-SLAM (visual-inertial SLAM) methods in which RS cameras or IMU data are typically used. The proposed synthetic dataset contains RS images, time-synchronized GS images, inertial measurement unit (IMU) measurements, and accurate ground truth. It provides camera images with 640×480 resolution at 30 Hz and IMU measurements from 90 Hz to 14400 Hz. The cubic B-spline curves are used to model the motion of trajectories. Based on the known trajectories, an image of each pose can be rendered, and the corresponding IMU measurement model is then established. The dataset provides realistic images and IMU measurements by modeling the sensor noise in RGB and IMU data. Two trajectories with three sequences of different motion speeds (i. e., slow, medium and fast corresponding to different rolling shutter effects) are contained in the proposed dataset. Herein, the proposed dataset can be applied to compare the impact of different rolling shutter effects on a specific method.

show abstract

Rolling-Shutter Modelling for Direct Visual-Inertial Odometry

Cited by 24 publications

References 25 publications

Continuous-Time Spatiotemporal Calibration of a Rolling Shutter Camera-IMU System

Continuous-Time Spatiotemporal Calibration of a Rolling Shutter Camera-IMU System

MBA-VO: Motion Blur Aware Visual Odometry

The WHU Rolling Shutter Visual-Inertial Dataset

Contact Info

Product

Resources

About