Vision-Centric BEV Perception: A Survey

Ma, Yuexin; Wang, Tai; Bai, Xuyang; Yang, Huitong; Hou, Yue; Wang, Yaming; Qiao, Yu; Yang, Ruigang; Manocha, Dinesh; Zhu, Xinge

doi:10.48550/arxiv.2208.02797

Cited by 6 publications

(8 citation statements)

References 83 publications

(138 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Projection-based methods generate dense voxel or BEV representation from image features through 3D-to-2D projection [1]. ImVoxelNet [16] aggregated the projected features from several images via a simple element-wise averaging, where spatial information might not be exploited sufficiently.…”

Section: B Camera-based Feature Fusionmentioning

confidence: 99%

“…Subject to sensor limitations, autonomous vehicles lack a global perception capability for monitoring holistic road conditions and accurately detecting surrounding objects, which bears great safety risks [1], [2]. Vehicle-to-everything (V2X) [3], [4] aims to build a communication system between vehicles and other devices in a complex traffic environment.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Beijing Language and Culture University, China

Wang¹

2022

The Experience of Examining the PhD

View full text Add to dashboard Cite

In the study of network synchronization, an outstanding question of both theoretical and practical significance is how to allocate a given set of heterogenous oscillators on a complex network in order for improving the synchronization performance. Whereas methods have been proposed to address this question in literature, the methods are based on accurate models describing the system dynamics, which, however, are normally unavailable in realistic situations. Here we show that this question can be addressed by the model-free technique of feed-forward neural network (FNN) in machine learning. Specifically, we measure the synchronization performance of a number of allocation schemes and use the measured data to train a machine. It is found that the trained machine is able to not only infer the synchronization performance of any new allocation scheme, but also find from a huge amount of candidates the optimal allocation scheme for synchronization.

show abstract

Section: B Camera-based Feature Fusionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Beijing Language and Culture University, China

Wang¹

2022

The Experience of Examining the PhD

View full text Add to dashboard Cite

show abstract

“…Online HD map construction can be seen as an advanced setting of lane detection, consisting of lines and polygons with various semantics in the local 360 • FOV perception range of egovehicle. With advanced 2D-to-BEV modules [30], previous online HD map construction methods cast it into semantic segmentation task on the transformed BEV features [27,38,19,28,26,29]. To build vectorized semantic HD map, HDMapNet [18] follows a segmentation-then-vectorization paradigm.…”

Section: Related Workmentioning

confidence: 99%

Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction

Liao¹,

Chen²,

Jiang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Online lane graph construction is a promising but challenging task in autonomous driving. Previous methods usually model the lane graph at the pixel or piece level, and recover the lane graph by pixel-wise or piece-wise connection, which breaks down the continuity of the lane. Human drivers focus on and drive along the continuous and complete paths instead of considering lane pieces. Autonomous vehicles also require path-specific guidance from lane graph for trajectory planning. We argue that the path, which indicates the traffic flow, is the primitive of the lane graph. Motivated by this, we propose to model the lane graph in a novel path-wise manner, which well preserves the continuity of the lane and encodes traffic information for planning. We present a path-based online lane graph construction method, termed LaneGAP, which endto-end learns the path and recovers the lane graph via a Path2Graph algorithm. We qualitatively and quantitatively demonstrate the superiority of LaneGAP over conventional pixel-based and piece-based methods. Abundant visualizations show LaneGAP can cope with diverse traffic conditions. Code and models will be released for facilitating future research.

show abstract

“…Projection-based methods generate dense voxel or BEV representation from image features through 3D-to-2D projection [19]. ImVoxelNet [25] aggregated the projected features from several images via a simple element-wise averaging, where spatial information might not be exploited sufficiently.…”

Section: Multi-view Camera Fusionmentioning

confidence: 99%

VIMI: Vehicle-Infrastructure Multi-view Intermediate Fusion for Camera-based 3D Object Detection

Wang¹,

Fan²,

Huo³

et al. 2023

Preprint

View full text Add to dashboard Cite

In autonomous driving, Vehicle-Infrastructure Cooperative 3D Object Detection (VIC3D) makes use of multi-view cameras from both vehicles and traffic infrastructure, providing a global vantage point with rich semantic context of road conditions beyond a single vehicle viewpoint. Two major challenges prevail in VIC3D: 1) inherent calibration noise when fusing multi-view images, caused by time asynchrony across cameras; 2) information loss when projecting 2D features into 3D space. To address these issues, We propose a novel 3D object detection framework, Vehicles-Infrastructure Multi-view Intermediate fusion (VIMI). First, to fully exploit the holistic perspectives from both vehicles and infrastructure, we propose a Multi-scale Cross Attention (MCA) module that fuses infrastructure and vehicle features on selective multi-scales to correct the calibration noise introduced by camera asynchrony. Then, we design a Camera-aware Channel Masking (CCM) module that uses camera parameters as priors to augment the fused features. We further introduce a Feature Compression (FC) module with channel and spatial compression blocks to reduce the size of transmitted features for enhanced efficiency. Experiments show that VIMI achieves 15.61% overall AP 3D and 21.44% AP BEV on the new VIC3D dataset, DAIR-V2X-C, significantly outperforming state-of-the-art early fusion and late fusion methods with comparable transmission cost.

show abstract

Vision-Centric BEV Perception: A Survey

Cited by 6 publications

References 83 publications

Beijing Language and Culture University, China

Beijing Language and Culture University, China

Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction

VIMI: Vehicle-Infrastructure Multi-view Intermediate Fusion for Camera-based 3D Object Detection

Contact Info

Product

Resources

About