Integrity monitoring of Graph‐SLAM using GPS and fish‐eye camera

Abstract

We propose a Simultaneous Localization and Mapping (SLAM)‐based Integrity Monitoring (IM) algorithm using GPS and fish‐eye camera to compute the protection levels while accounting for multiple faults in GPS and vision. We perform graph optimization using GPS pseudoranges, pixel intensities, vehicle dynamics, and satellite ephemeris to simultaneously localize the vehicle, GPS satellites, and key image pixels in the world frame. We estimate the fault mode vector by analyzing the temporal correlation across pseudorange residuals and spatial correlation across pixel intensity residuals. To isolate the vision faults, we develop a superpixel‐based piecewise random sample consensus. For the estimated fault mode, we compute the protection levels by performing worst‐case failure slope analysis on the batch realization of linearized Graph‐SLAM formulation. We perform real‐world experiments in an alleyway in Stanford, California and a semi‐urban area in Champaign, Illinois. We demonstrate higher localization accuracy and tighter protection levels as compared to GPS‐only SLAM‐based IM.

Robust GPS-Vision Localization via Integrity-Driven Landmark Attention

“…However, analyzing vision faults requires access to a reference map database, and therefore, the reliability of this approach depends on the quality and accuracy of the map considered. In our prior works (Bhamidipati & Gao, 2019, 2020a, we performed FDE using analysis of temporal correlation across GPS measurement residuals and spatial correlation across vision intensity residuals. While these methodologies tackled both GPS and vision faults, the FDE analysis was conducted in the measurement domain, and moreover, separate FDE approaches are still to be designed for GPS and vision.…”

“…We considered an off-the-shelf estimator from the existing literature (Bhamidipati & Gao, 2020a;Shepard & Humphreys, 2014) on GPS-vision localization in urban areas. The off-the-shelf GPS-vision estimator considers the same measurement models as that of the proposed ILA technique discussed earlier in Section 3.1.…”

“…As seen in Figure 8, the initialization procedure of the proposed ILA technique was performed in two stages: camera calibration and GPS-vision calibration. Referring to the existing works (Bhamidipati & Gao, 2020a;Engel et al, 2018), we followed a standard calibration procedure for the camera comprised of intrinsic calibration, removal of radial distortion, and photometric calibration. Using several images taken from different positions, we estimated the intrinsic parameters (i.e., f x , f y , c x , and c y ) that mapped the 2D image frame to the 3D coordinates of the camera frame, and vice-versa, which are denoted by π and π −1 in Equation ( 7), respectively.…”

“…This step is conducted in fairly open-sky settings (i.e., no simulated multipath in GPS measurements) with the presence of feature-rich conditions for the camera, such as a fence, billboards, trees, etc. Referring to Bhamidipati and Gao (2020a) and Chen et al (2018), we set the global reference frame to be the GPS frame at the start of the calibration step, which denoted the East-North-Up (ENU) coordinate system whose center is fixed to the receiver location.…”

“…To resolve the scale factor, we first independently estimated the system positions in the global reference frame of the executed random motion via two variants of the off-the-shelf GPS-vision estimator (Bhamidipati & Gao, 2020a;Engel et al, 2018): one using GPS-only measurements, and the other using vision-only measurements. For vision-only measurements, we estimated the system positions in the global reference frame by setting an initial scale factor of one and using the parameters computed from camera calibration, which is the first stage of initialization, and extrinsic calibration, which was explained earlier.…”