End-to-end Lane Detection through Differentiable Least-Squares Fitting

Abstract

Lane detection is typically tackled with a two-step pipeline in which a segmentation mask of the lane markings is predicted first, and a lane line model like a parabola or spline is fitted to the post-processed mask next. The problem with such a two-step approach is that the parameters of the network are not optimized for the true task of interest (estimating the lane curvature parameters) but for a proxy task (segmenting the lane markings), resulting in sub-optimal performance. In this work, we propose a method to train a lane detector in an end-to-end manner, directly regressing the lane parameters. The architecture consists of two components: a deep network that predicts a segmentationlike weight map for each lane line, and a differentiable least-squares fitting module that returns for each map the parameters of the best-fitting curve in the weighted leastsquares sense. These parameters can subsequently be supervised with a loss function of choice. Our method relies on the observation that it is possible to backpropagate through a least-squares fitting procedure. This leads to an end-to-end method where the features are optimized for the true task of interest: the network implicitly learns to generate features that prevent instabilities during the model fitting step, as opposed to two-step pipelines that need to handle outliers with heuristics. Additionally, the system is not just a black box but offers a degree of interpretability because the intermediately generated segmentation-like weight maps can be inspected and visualized. Code and a video is available at github.com/wvangansbeke/ LaneDetection_End2End.

A Survey on Deep Learning Based Approaches for Scene Understanding in Autonomous Driving

“…The method of end-to-end lane segmentation based on deep learning has fewer corresponding research results. For example, Van et al [94] proposed a method to train a lane detector in an end-to-end manner, directly regressing the lane parameters. The architecture consisted of two components: a deep network that predicted a segmentation-like weight map for each lane line and a differentiable least squares fitting module that returned the parameters of the best fitting curve in the weighted least squares sense for each map.…”

Deep Learning in Lane Marking Detection: A Survey

“…2) Regression losses: When the output of a deep learner is expected to be continuous, regression losses are more suitable compared with those classification ones mentioned in section III-A1. In lane marking detection, the most commonly used regression losses are the mean squared error (MSE) [53], [59], [64], [75], [77]- [79], mean absolute error (MAE) [54], [79], [80], and Huber loss defined as…”

A Region-Based Efficient Network for Accurate Object Detection

“…Numerous techniques are now available to achieve state-of-the-art detection performance. Recently, object detection has been extensively used in various fields, such as medicine [41], roads, building detection [42,43], automatic detection of lane marking [44], face detection [45], and pedestrian detection [46].…”