Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback

Abstract

This paper presents a visual-inertial odometry framework which tightly fuses inertial measurements with visual data from one or more cameras, by means of an iterated extended Kalman filter (IEKF). By employing image patches as landmark descriptors, a photometric error is derived, which is directly integrated as an innovation term in the filter update step. Consequently, the data association is an inherent part of the estimation process and no additional feature extraction or matching processes are required. Furthermore, it enables the tracking of non-corner shaped features, such as lines, and thereby increases the set of possible landmarks. The filter state is formulated in a fully robocentric fashion, which reduces errors related to nonlinearities. This also includes partitioning of a landmark's location estimate into a bearing vector and distance and thereby allows an undelayed initialization of landmarks. Overall, this results in a compact approach which exhibits a high level of robustness with respect to low scene texture and motion blur. Furthermore, there is no time-consuming initialization procedure and pose estimates are available starting at the second image frame. We test the filter on different real datasets and compare it to other state-of-the-art visual-inertial frameworks. The experimental results show that robust localization with high accuracy can be achieved with this filterbased framework.

Direct Visual-Inertial Ego-Motion Estimation Via Iterated Extended Kalman Filter

“…where α is the inverse orthogonal distance (α = d −1 ) from the camera center to the surface. The inverse parameterization has been shown to produce superior accuracy in [16]. Similar to [18], ϑ := v/d ∈ R 3 is defined as the ratio of flight velocity to the distance.…”

“…As illustrated in Fig. 2, the proposed method differs from the work [16] owing to the single-plane assumption. The simplification means there exists only a low-dimension state vector associated with the image measurement model and the continuous homography equation.…”

“…The observer is unable to A sketch comparing the measurement model of the patch-based approach [16] with the proposed method: (a) the patch-based method in [16] and (b) the proposed method. In [16], an intensity patch around each image feature in the reference frame Ir is warped into the current frame Ic according to the feature's depth and relative pose between two frames provided by the IMU prediction. The difference between the warped patch and the actual measurement constitutes the innovation term to update the state vector.…”

“…The proposed framework takes motivation from the previous indirect reactive navigation method [11] and the semi-direct use of photometric feedback through IEKF [16]. That is, the photometric error from an entire image is directly integrated into the IMU measurements for the ego-motion estimation via the IEKF framework under the assumption of a single planar scene.…”

“…This makes the complex data association and mapping process unnecessary. To enhance the robustness, each pixel value on the image is integrated into Kalman update step instead of the multiple patches around the points of interest as in [16]. To deal with the large observation vector comprising of pixel intensities from the entire image, a Gauss-Newton Kalman gain [19] is used to substantially reduce the computation complexity.…”