Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systems

Abstract

Predicting the future location of vehicles is essential for safety-critical applications such as advanced driver assistance systems (ADAS) and autonomous driving. This paper introduces a novel approach to simultaneously predict both the location and scale of target vehicles in the first-person (egocentric) view of an ego-vehicle. We present a multi-stream recurrent neural network (RNN) encoder-decoder model that separately captures both object location and scale and pixellevel observations for future vehicle localization. We show that incorporating dense optical flow improves prediction results significantly since it captures information about motion as well as appearance change. We also find that explicitly modeling future motion of the ego-vehicle improves the prediction accuracy, which could be especially beneficial in intelligent and automated vehicles that have motion planning capability. To evaluate the performance of our approach, we present a new dataset of first-person videos collected from a variety of scenarios at road intersections, which are particularly challenging moments for prediction because vehicle trajectories are diverse and dynamic. Code and dataset have been made available at: https: //usa.honda-ri.com/hevi arXiv:1809.07408v2 [cs.CV]

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

“…Fan et al [13] and Luc et al [30] learned feature to feature translation to forecast features into the future. To exploit the time dependency inherent in future prediction, many works use RNNs and LSTMs [58,48,50,54,49]. Liu et al [29] and Rybkin et al [45] formulated the translation from two consecutive images in a video by an autoencoder to infer the next frame.…”

Looking to Relations for Future Trajectory Forecast

“…Such technologies require advanced decision making and motion planning systems that rely on estimates of the future position of road users in order to realize safe and effective mitigation and navigation strategies. Related research [46,1,36,23,37,12,13,43,45,32,33,47] has attempted to predict future trajectories by focusing on social conventions, environmental factors, or pose and motion constraints. They have shown to be more effective when the prediction model learns to extract these features by considering human-human (i.e., between road agents) or human-space (i.e., between a road agent and environment) interactions.…”

Unsupervised Traffic Accident Detection in First-Person Videos

“…Yagi et al [17] incorporate different kinds of cues into a convolution-deconvolution (Conv1D) network to predict pedestrians' future locations. Yao et al [18] extend this work to autonomous driving scenarios by proposing a multi-stream RNN Encoder-Decoder (RNN-ED) architecture with both past vehicle locations and image features as inputs for anticipating vehicle locations.…”