Laneformer: Object-Aware Row-Column Transformers for Lane Detection

Abstract

We present Laneformer, a conceptually simple yet powerful transformer-based architecture tailored for lane detection that is a long-standing research topic for visual perception in autonomous driving. The dominant paradigms rely on purely CNN-based architectures which often fail in incorporating relations of long-range lane points and global contexts induced by surrounding objects (e.g., pedestrians, vehicles). Inspired by recent advances of the transformer encoder-decoder architecture in various vision tasks, we move forwards to design a new end-to-end Laneformer architecture that revolutionizes the conventional transformers into better capturing the shape and semantic characteristics of lanes, with minimal overhead in latency. First, coupling with deformable pixel-wise self-attention in the encoder, Laneformer presents two new row and column self-attention operations to efficiently mine point context along with the lane shapes. Second, motivated by the appearing objects would affect the decision of predicting lane segments, Laneformer further includes the detected object instances as extra inputs of multi-head attention blocks in the encoder and decoder to facilitate the lane point detection by sensing semantic contexts. Specifically, the bounding box locations of objects are added into Key module to provide interaction with each pixel and query while the ROI-aligned features are inserted into Value module. Extensive experiments demonstrate our Laneformer achieves state-of-the-art performances on CULane benchmark, in terms of 77.1% F1 score. We hope our simple and effective Laneformer will serve as a strong baseline for future research in self-attention models for lane detection.

An Enhanced Aircraft Carrier Runway Detection Method Based on Image Dehazing

“…Such as, CLRNet [31] employs learnable anchor parameters, started from x 0 , y 0 , θ 0 , and runway length l 0 . LaneFormer [32] introduces a novel Transformer with x and y axis attention mechanism to detect runway instances. UFLD [5] leverages the flattening operation to convert the 2D feature map into a 1D vector, and then the row-wise runway point positions are detected by the 1D feature map.…”

Lane Detection with Deep Learning: Methods and Datasets

“…A lane detection method with a row-column self-attention module using a transformer structure is proposed in [7]. The input image is first passed through a ResNet backbone to obtain row and column features.…”

“…Among these methods, the most widely used are end-to-end networks with convolutional encoders and decoders [21,32,33]. Some studies improve the structure of convolution to obtain better results [6,25,32], while different attention mechanisms are introduced in [4,7,29,33] to achieve certain progress. Emerging graph neural networks have also been applied [10], and the processing of continuous images is also under study [4].…”