Laneformer: Object-aware Row-Column Transformers for Lane Detection

Abstract

We present Laneformer, a conceptually simple yet powerful transformer-based architecture tailored for lane detection that is a long-standing research topic for visual perception in autonomous driving. The dominant paradigms rely on purely CNN-based architectures which often fail in incorporating relations of long-range lane points and global contexts induced by surrounding objects (e.g., pedestrians, vehicles). Inspired by recent advances of the transformer encoder-decoder architecture in various vision tasks, we move forwards to design a new end-to-end Laneformer architecture that revolutionizes the conventional transformers into better capturing the shape and semantic characteristics of lanes, with minimal overhead in latency. First, coupling with deformable pixel-wise selfattention in the encoder, Laneformer presents two new row and column self-attention operations to efficiently mine point context along with the lane shapes. Second, motivated by the appearing objects would affect the decision of predicting lane segments, Laneformer further includes the detected object instances as extra inputs of multi-head attention blocks in the encoder and decoder to facilitate the lane point detection by sensing semantic contexts. Specifically, the bounding box locations of objects are added into Key module to provide interaction with each pixel and query while the ROI-aligned features are inserted into Value module. Extensive experiments demonstrate our Laneformer achieves state-of-the-art performances on CULane benchmark, in terms of 77.1% F1 score. We hope our simple and effective Laneformer will serve as a strong baseline for future research in self-attention models for lane detection.

BSNet: Lane Detection via Draw B-spline Curves Nearby

“…LSTR [14] utilizes DETR-like [4] structure and geometry constraints of lanes to improve detection performance. Laneformer [7] uses a transformer encoder and tries to exploit the relationship between occluded parts of lanes and vehicles. Under the guidance of the vehicle's position information, the occluded lanes are completed.…”

CurveFormer: 3D Lane Detection by Curve Propagation with Curve Queries and Attention

“…It detects lanes in the image plane and projects them to 3D space with camera pose. In general, advanced monocular lane detectors can be categorized into segmentation approaches [1]- [4] and regression approaches [5]- [12], [27]. SCNN [1] is proposed to propagate context by slice-byslice convolutions within feature maps.…”

“…Lane regression algorithms can be grouped into key points estimation [5], [6], anchor-based regression [7]- [11] and row-wise regression [12], [27]. PINet [5] combines key points estimation and instance segmentation, and GANet [6] represents lanes as a set of key points which are only related to the start point.…”

“…Ultra-Fast [12] introduces a novel row-wise classification method with remarkable speed. Laneformer [27] applies row-column self-attentions to accommodate the conventional Transformer to capture the shape characteristics and semantic contexts of lanes.…”

Analyzing Performance Effects of Neural Networks Applied to Lane Recognition under Various Environmental Driving Conditions

“…It also demonstrated good adaptability to scenarios with nighttime conditions or with occluded parts; however, it does not address complex lane detection tasks. Likewise, in [41], Han et al developed the Laneformer transformer-based architecture and adapted it to lane detection. This better captures the lane shape characteristics and the global semantic context of the series of points defining the lanes using a row and column selfattenuation mechanism.…”