Robust Deep-Learning-Based Road-Prediction for Augmented Reality Navigation Systems at Night

Abstract

This paper proposes an approach that predicts the road course from camera sensors leveraging deep learning techniques. Road pixels are identified by training a multi-scale convolutional neural network on a large number of full-scene-labeled nighttime road images including adverse weather conditions. A framework is presented that applies the proposed approach to longer distance road course estimation, which is the basis for an augmented reality navigation application. In this framework long range sensor data (radar) and data from a map database are fused with short range sensor data (camera) to produce a precise longitudinal and lateral localization and road course estimation. The proposed approach reliably detects roads with and without lane markings and thus increases the robustness and availability of road course estimations and augmented reality navigation. Evaluations on an extensive set of high precision ground truth data taken from a differential GPS and an inertial measurement unit show that the proposed approach reaches state-of-the-art performance without the limitation of requiring existing lane markings. This work was partially funded by the European Commission under the ECSEL Joint Undertaking in the scope of the DESERVE project. http://www.deserve-project.eu/ 1 M. Limmer, J. Forster, D. Baudach and R. Schweiger are with

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

“…In this context four representative works are selected. The first framework [52] applies CNNs to estimate longer distance road course for augmented reality applications. The second one investigates cascaded end-to-end CNN (CasNet) for accurate road detection and localization of centerline in the presence of complex backgrounds and significant occlusions of trees and cars as given in [53].…”

Review of Key Technologies and Progress in Industrial Equipment Health Management

“…The data-driven method mainly adopts machine learning or statistical methods. As a new method in the field of machine learning [14], deep learning [15] has made brilliant achievements in the fields of image and speech recognition due to its strong automatic feature extraction ability. As one of the classic algorithms of deep learning, the deep belief network (DBN) has successfully solved problems such as information retrieval, dimension reduction and fault classification with its excellent feature extraction and training algorithm.…”

Improved IR-Colorization using Adversarial Training and Estuary Networks

“…To maintain the input resolution, but also leveraging the encoding effect, Fragmentation Layers [10,41,42] can be used. Numerous network architectures [4,26,27,34,46] utilize this technique in a variation commonly known as dilated convolution. The common practice, however, is to keep the encoding structure and to design an additional decoder part for the network.…”

“…While Encoder-Decoder networks implicitly handle multi-scalar processing of the input, other approaches explicitly handle multi-scalar input, e.g. [7,9,26,27,45]. These ap- proaches construct an image pyramid with regular downsampling operations before processing every scale concurrently in their own branch of the network.…”

“…The outputs of each branch are finally combined after they have been upsampled to the same resolution. Approaches like [7,45] train the network branch by branch, while [9,26,27] train the network with all branches end to end. Results by [26] show that multiple scales strongly benefit the network's recall performance.…”