Street-View Image Generation from a Bird's-Eye View Layout

Abstract

Bird's-Eye View (BEV) Perception has received increasing attention in recent years as it provides a concise and unified spatial representation across views and benefits a diverse set of downstream driving applications. While the focus has been placed on discriminative tasks such as BEV segmentation, the dual generative task of creating streetview images from a BEV layout has rarely been explored. The ability to generate realistic street-view images that align with a given HD map and traffic layout is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving. In this paper, we propose BEVGen, a conditional generative model that synthesizes a set of realistic and spatially consistent surrounding images that match the BEV layout of a traffic scenario. BEVGen incorporates a novel cross-view transformation and spatial attention design which learn the relationship between cameras and map views to ensure their consistency. Our model can accurately render road and lane lines, as well as generate traffic scenes under different weather conditions and times of day. The code will be made publicly available.

Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion

“…A set of recent, unpublished, and concurrent works discuss 3D street view generation Swerdlow et al 2023;]. These methods take a bird's eye view-a layout map with each object in the scene-and ground truth 3D bounding boxes of all objects as input to generate street view images.…”

FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems

“…BEV perception research has begun exploring the use of multi-view camera data for predicting BEV maps due to their low cost [9]- [13]. Nevertheless, inferring 3D information from 2D data is challenging, as mono cameras only provide 2D information.…”