Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

Abstract

Vehicle and surrounding environment dynamic analysis (VSEDA) is an indispensable component of modern assistive drivings. A robust and accurate VSEDA could ensure the driving system reliability in presence of highly dynamic environments. This paper proposes a novel VSEDA framework by fusing the measurements from an inertial sensor and a monocular camera. Compared to traditional visualinertial-based assistive driving methods, the proposed approach can analyze both the vehicle dynamics and the surrounding environment. Even in the scenario that moving objects occupy a majority area of the scene captured in the image, the proposed method can still robustly analyze the surrounding environment by identifying the static inliers and dynamic inliers, which lie on stationary objects and moving objects, respectively. The theoretical framework consists of three steps. First, the vehicle nonholonomic constraint is applied to pairwise feature matching. For vehicle dynamic analysis, the static inliers are selected by choosing the features with their histogram bins consistent with inertial orientations. Second, for the surrounding environment dynamic analysis, the dynamic inliers are matched through histogram voting, together with the developed part-based vehicle detection model that can segment and match the vehicle regions from the background in image pairs. Finally, both the vehicle dynamics and surrounding environments are analyzed with static and dynamic inliers respectively. The proposed method has been evaluated on the challenging datasets, part of which was collected during rush hours in downtown areas. The experimental results prove the effectiveness and accuracy of the proposed VSEDA. INDEX TERMS Vehicle and surrounding environment dynamic analysis (VSEDA), assistive driving, monocular camera, inertial measurement unit (IMU), multi-sensor fusion, complex road conditions.

Enhanced Indoor Visual Navigation using Sensor Fusion and Semantic Information

“…Using non-holonomic constraints can decrease the system's complexity and increase the solution's accuracy. In contrast to some existing fusion schemes, such as those in [120], the non-holonomic motion constraints enabled the fusion of speed information obtained from processing stereo vision sensors. Using vision-estimated single-dimensional forward speed instead of a 3D position has shown more robustness against visual positioning errors.…”

“…The developed system has been demonstrated in a real indoor environment using a stereo-camera sensor, a builtin IMU sensor, and a local wireless infrastructure of the UWB radio network. In previous work [89], [88], conventional EKF designs and updates are taken as absolute positions calculated by stereo vision [121], [120]. However, this technique is not robust against visual tracking errors.…”