Real-to-Synthetic: Generating Simulator Friendly Traffic Scenes from Graph Representation

Tian, Yafu; Carballo, Alexander; Li, Ruifeng; Takeda, Kazuya

doi:10.1109/iv51971.2022.9827441

Cited by 7 publications

(11 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early work of Ulbrich et al [15] proposes an ontology for representing a scene graph for autonomous driving, but do not provide means for reasoning on the graph. Tian et al [16] propose a simplified approach by not modeling lanes explicitly. This is a limitation compared to our work because their approach cannot capture topological nor regulatory relationships between lanes.…”

Section: Graph-based Approachesmentioning

confidence: 99%

SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks

Monninger

Schmidt

Rupprecht

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. In this work we propose SCENE, a methodology to encode diverse traffic scenes in heterogeneous graphs and to reason about these graphs using a heterogeneous Graph Neural Network encoder and task-specific decoders. The heterogeneous graphs, whose structures are defined by an ontology, consist of different nodes with type-specific node features and different relations with type-specific edge features. In order to exploit all the information given by these graphs, we propose to use cascaded layers of graph convolution. The result is an encoding of the scene. Task-specific decoders can be applied to predict desired attributes of the scene. Extensive evaluation on two diverse binary node classification tasks show the main strength of this methodology: despite being generic, it even manages to outperform task-specific baselines. The further application of our methodology to the task of node classification in various knowledge graphs shows its transferability to other domains.

show abstract

Section: Graph-based Approachesmentioning

confidence: 99%

SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks

Monninger

Schmidt

Rupprecht

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…To obtain such relationships, we first convert the nuScenes map to the ASAM OpenDrive [69] format. This process is described in detail in our previous work [2]. By constructing high-precision maps on the nuScenes dataset, we can obtain the connections and "Belongs-to" relationships between map components.…”

Section: A Data Setupmentioning

confidence: 99%

“…Its node feature matrix consists of the actor's current feature vector α = [c i , (x i , y i ), (b w , b l , b θ ), (v x , v y )]. For the map-to-map layer subgraph G M M ,, we first obtain an OpenDRIVE format HD map from the nuScenes dataset, using a method we've proposed previously called "Real-to-synthetic" [71]. We also obtain (1) the driving directions of the roads and lanes;…”

Section: B Graph Pre-processingmentioning

confidence: 99%

“…Here, we briefly introduce our previous work on RSG-based synthetic traffic scene generation [71]. The purpose of this work is to automatically generate digital twins of open-source traffic scene dataset.…”

Section: E Application Of Rsg: Synthetic Traffic Scene Generationmentioning

confidence: 99%

“…By constructing an RSG graph that contains semantic relationships as described in this paper, combined with a graph autoencoder network, we can generate multiple similar and different traffic scenes, and simulate them in a simulator. Relevant work is detailed in our paper [2].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RSG-GCN: Predicting Semantic Relationships in Urban Traffic Scene With Map Geometric Prior

Tian

Carballo

et al. 2023

IEEE Open J. Intell. Transp. Syst.

Self Cite

View full text Add to dashboard Cite

Automated identification of the relationships between traffic actors and surrounding objects, in order to describe their behavior and predict their intentions, has become the focus of increasing attention in the field of autonomous driving. Therefore, in this work, we propose a Road Scene Graphs-Graph Convolutional Network (RSG-GCN) as a novel, graphbased model for predicting the topological graph structure of a given traffic scene. The status of the actors and HD map information are integrated as prior knowledge, allowing the edges linking the actor nodes to capture potential semantic relationships, such as "vehicle approaching pedestrian" and "pedestrian waiting at intersection". To train this model, we created our own RSG dataset, as well as a relational dataset and benchmark derived from nuScenes. Our extensive range of experiments demonstrate that our model can more accurately predict semantic relationships and behavior in a given traffic scene than other popular traffic scene prediction models. In particular, regarding the use of HD map prior knowledge, we found that the resulting increase in accuracy significantly outweighs performance loss caused by the increase in graph size. The downstream applications of RSG include traffic scene retrieval and synthetic traffic scene generation, which are briefly described.

show abstract

From a Visual Scene to a Virtual Representation: A Cross-Domain Review

et al. 2023

View full text Add to dashboard Cite

The widespread use of smartphones and other low-cost equipment as recording devices, the massive growth in bandwidth, and the ever-growing demand for new applications with enhanced capabilities, made visual data a must in several scenarios, including surveillance, sports, retail, entertainment, and intelligent vehicles. Despite significant advances in analyzing and extracting data from images and video, there is a lack of solutions able to analyze and semantically describe the information in the visual scene so that it can be efficiently used and repurposed. Scientific contributions have focused on individual aspects or addressing specific problems and application areas, and no cross-domain solution is available to implement a complete system that enables information passing between cross-cutting algorithms. This paper analyses the problem from an end-to-end perspective, i.e., from the visual scene analysis to the representation of information in a virtual environment, including how the extracted data can be described and stored. A simple processing pipeline is introduced to set up a structure for discussing challenges and opportunities in different steps of the entire process, allowing to identify current gaps in the literature. The work reviews various technologies specifically from the perspective of their applicability to an endto-end pipeline for scene analysis and synthesis, along with an extensive analysis of datasets for relevant tasks.

show abstract

Real-to-Synthetic: Generating Simulator Friendly Traffic Scenes from Graph Representation

Cited by 7 publications

References 27 publications

SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks

SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks

RSG-GCN: Predicting Semantic Relationships in Urban Traffic Scene With Map Geometric Prior

From a Visual Scene to a Virtual Representation: A Cross-Domain Review

Contact Info

Product

Resources

About