Fast, robust, continuous monocular egomotion computation

Jaegle, Andrew; Phillips, Stephen; Daniilidis, Kostas

doi:10.1109/icra.2016.7487206

Cited by 23 publications

(28 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Joint estimation of object and ego motion from monocular RGB frames can be ambiguous [4]. However, the estimation of ego-and object-motion components from their composite optic flow could be improved by using the geometric constraints of the motion field to regularize a deep neural network-based predictor [19], [29].…”

Section: Introductionmentioning

confidence: 99%

“…Compared with the existing approaches, our method does not assume a static scene [15], [16], [31] and does not require dynamic segment mask [2], [6], [26] or depth [11], [27], [28] for ego-motion prediction from monocular RGB frames. This is achieved by using continuous ego-motion constraints to train a neural network-based predictor, which allows the network to remove variations due to depth and moving objects in the input frames [19], [29]. Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sparse Representations for Object- and Ego-Motion Estimations in Dynamic Scenes

Kashyap

Fowlkes²,

Krichmar³

2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Disentangling the sources of visual motion in a dynamic scene during self-movement or ego motion is important for autonomous navigation and tracking. In the dynamic image segments of a video frame containing independently moving objects, optic flow relative to the next frame is the sum of the motion fields generated due to camera and object motion. The traditional ego-motion estimation methods assume the scene to be static, and the recent deep learning-based methods do not separate pixel velocities into object-and ego-motion components. We propose a learning-based approach to predict both ego-motion parameters and object-motion field (OMF) from image sequences using a convolutional autoencoder while being robust to variations due to the unconstrained scene depth. This is achieved by: 1) training with continuous ego-motion constraints that allow solving for ego-motion parameters independently of depth and 2) learning a sparsely activated overcomplete ego-motion field (EMF) basis set, which eliminates the irrelevant components in both static and dynamic segments for the task of ego-motion estimation. In order to learn the EMF basis set, we propose a new differentiable sparsity penalty function that approximates the number of nonzero activations in the bottleneck layer of the autoencoder and enforces sparsity more effectively than L1-and L2-norm-based penalties. Unlike the existing direct ego-motion estimation methods, the predicted global EMF can be used to extract OMF directly by comparing it against the optic flow. Compared with the state-of-the-art baselines, the proposed model performs favorably on pixelwise object-and ego-motion estimation tasks when evaluated on real and synthetic data sets of dynamic scenes.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Sparse Representations for Object- and Ego-Motion Estimations in Dynamic Scenes

Kashyap

Fowlkes²,

Krichmar³

2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…The images of surrounding environment captured by the vehicle vision system provide abundant surrounding traffic information, such as static inliers (correct feature matches on static scenes) and dynamic inliers (correct feature matches on moving vehicles). Recent works have extensively leveraged various visual feature attributes, (e.g., lane detection [9], feature matching [10], semantic segmentation [11]) to compute vehicle dynamics and simultaneously guide the vehicle on road. It is notable that Scaramuzza [12] has applied the nonholonomic constraint model to estimate ego motion of ground vehicles.…”

Section: Introductionmentioning

confidence: 99%

Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

Zhang

Liang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Vehicle and surrounding environment dynamic analysis (VSEDA) is an indispensable component of modern assistive drivings. A robust and accurate VSEDA could ensure the driving system reliability in presence of highly dynamic environments. This paper proposes a novel VSEDA framework by fusing the measurements from an inertial sensor and a monocular camera. Compared to traditional visualinertial-based assistive driving methods, the proposed approach can analyze both the vehicle dynamics and the surrounding environment. Even in the scenario that moving objects occupy a majority area of the scene captured in the image, the proposed method can still robustly analyze the surrounding environment by identifying the static inliers and dynamic inliers, which lie on stationary objects and moving objects, respectively. The theoretical framework consists of three steps. First, the vehicle nonholonomic constraint is applied to pairwise feature matching. For vehicle dynamic analysis, the static inliers are selected by choosing the features with their histogram bins consistent with inertial orientations. Second, for the surrounding environment dynamic analysis, the dynamic inliers are matched through histogram voting, together with the developed part-based vehicle detection model that can segment and match the vehicle regions from the background in image pairs. Finally, both the vehicle dynamics and surrounding environments are analyzed with static and dynamic inliers respectively. The proposed method has been evaluated on the challenging datasets, part of which was collected during rush hours in downtown areas. The experimental results prove the effectiveness and accuracy of the proposed VSEDA. INDEX TERMS Vehicle and surrounding environment dynamic analysis (VSEDA), assistive driving, monocular camera, inertial measurement unit (IMU), multi-sensor fusion, complex road conditions.

show abstract

“…Vision sensors have been commonly used for vehicle and traffic dynamics analysis such as lane detection [3], feature matching [4], and semantic segmentation [5]. It is remarkable that Scaramuzza et al [6] has applied the nonholonomic constraint model to estimate ground vehicle motions.…”

Section: Introductionmentioning

confidence: 99%

Perception of Vehicle and Traffic Dynamics Using Visual-Inertial Sensors for Assistive Driving

Zhang

Liang

et al. 2018

2018 IEEE International Conference on Robotics and Biomimetics (ROBIO)

View full text Add to dashboard Cite

This paper presents an approach to analyze vehicle and traffic dynamics by fusing a monocular camera and inertial sensors. As opposed to traditional visual-inertial odometry for ground vehicles, the proposed method can estimate both the dynamics of the vehicle and the dynamics of the surrounding environment of the vehicle. The visual features associated with the surrounding environment are determined by the nonholonomic constraint and inertial measurements of the vehicle, and the visual features associated with moving vehicles are segmented by a part-based vehicle detection model. The dynamics of the vehicle and the scene are computed from respective visual features. The proposed method is robust to high-dynamic environment such as the scenario of many moving vehicles during rush hours in downtown areas. In addition, the proposed method is capable of estimating the number of surrounding vehicles as well as the ratios of vehicle regions to the whole image area. Experiments were performed on a challenging dataset that was collected in downtown areas and interstate highways during rush hours. The experimental results showed that the proposed method can robustly and accurately analyze the dynamics of vehicles and surrounding environments.

show abstract

Fast, robust, continuous monocular egomotion computation

Cited by 23 publications

References 37 publications

Sparse Representations for Object- and Ego-Motion Estimations in Dynamic Scenes

Sparse Representations for Object- and Ego-Motion Estimations in Dynamic Scenes

Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

Perception of Vehicle and Traffic Dynamics Using Visual-Inertial Sensors for Assistive Driving

Contact Info

Product

Resources

About