A new performance measure and evaluation benchmark for road detection algorithms

Abstract

Abstract-Detecting the road area and ego-lane ahead of a vehicle is central to modern driver assistance systems. While lane-detection on well-marked roads is already available in modern vehicles, finding the boundaries of unmarked or weakly marked roads and lanes as they appear in inner-city and rural environments remains an unsolved problem due to the high variability in scene layout and illumination conditions, amongst others. While recent years have witnessed great interest in this subject, to date no commonly agreed upon benchmark exists, rendering a fair comparison amongst methods difficult.In this paper, we introduce a novel open-access dataset and benchmark for road area and ego-lane detection. Our dataset comprises 600 annotated training and test images of high variability from the KITTI autonomous driving project, capturing a broad spectrum of urban road scenes. For evaluation, we propose to use the 2D Bird's Eye View (BEV) space as vehicle control usually happens in this 2D world, requiring detection results to be represented in this very same space. Furthermore, we propose a novel, behavior-based metric which judges the utility of the extracted ego-lane area for driver assistance applications by fitting a driving corridor to the road detection results in the BEV. We believe this to be important for a meaningful evaluation as pixel-level performance is of limited value for vehicle control. State-of-the-art road detection algorithms are used to demonstrate results using classical pixellevel metrics in perspective and BEV space as well as the novel behavior-based performance measure. All data and annotations are made publicly available on the KITTI online evaluation website in order to serve as a common benchmark for road terrain detection algorithms.

Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding

“…For the KITTI Vision road detection benchmark, performance is measured in the birds-eye view, while data is presented in ego view. The authors of (Fritsch et al, 2013) claim that the vehicle control usually happens in 2D space and therefore road detection should also be done in this space. A wrong classified pixel near the horizon in ego view represents a whole bunch of pixels in the birds-eye view.…”

Review of Lane Detection and Tracking Algorithms in Advanced Driver Assistance System

“…The most common metrics used for evaluating performance of lane detection algorithms are Precision, Recall, F-score, Accuracy [34], Receiver Operating Characteristic (ROC) curves and Dice Similarity Coefficient (DSC) [33]. Precision is the fraction of detected lanes markers that are actual lane markers.…”

Sensor Fault Detection and Diagnosis for autonomous vehicles

“…The implementation of the perception system has been done based on the perception sensors available in the KITTI dataset [19][20][21], which includes a Velodyne sensor and two pairs of stereo vision cameras. The federated perception architecture suggested to fuse sensor data from the KITTI dataset is shown in Figure 3 …”