A Knowledge Graph-Based Approach for Situation Comprehension in Driving Scenarios

Halilaj, Lavdim; Dindorkar, Ishan; Lüttin, Jürgen; Rothermel, Susanne

doi:10.1007/978-3-030-77385-4_42

Cited by 22 publications

(9 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of KGs in the AD domain has not received too much attention at the current point of time, albeit it can be an effective way to help situation or scene understanding [732]. For instance, the authors of [283] built a specific ontology to represent all core concepts that are essential to model the driving concept. The built KG CoSi models information about driver, vehicle, road infrastructure, driving situation and interacting traffic participants [283].…”

Section: Applicationsmentioning

confidence: 99%

“…For instance, the authors of [283] built a specific ontology to represent all core concepts that are essential to model the driving concept. The built KG CoSi models information about driver, vehicle, road infrastructure, driving situation and interacting traffic participants [283]. To classify the underlying traffic situation with a NN, a relational graph convolutional network [637] is used to convert the underlying KG into embeddings first.…”

Section: Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

Wörmann¹,

Bogdoll²,

Bührle³

et al. 2022

Preprint

View full text Add to dashboard Cite

The existence of representative datasets is a prerequisite of many successful artificial intelligence and machine learning models. However, the subsequent application of these models often involves scenarios that are inadequately represented in the data used for training. The reasons for this are manifold and range from time and cost constraints to ethical considerations. As a consequence, the reliable use of these models, especially in safety-critical applications, is a huge challenge. Leveraging additional, already existing sources of knowledge is key to overcome the limitations of purely data-driven approaches, and eventually to increase the generalization capability of these models. Furthermore, predictions that conform with knowledge are crucial for making trustworthy and safe decisions even in underrepresented scenarios. This work provides an overview of existing techniques and methods in the literature that combine data-based models with existing knowledge. The identified approaches are structured according to the categories integration, extraction and conformity. Special attention is given to applications in the field of autonomous driving.Acknowledgement: The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Climate Action within the project "KI Wissen -Entwicklung von Methoden für die Einbindung von Wissen in maschinelles Lernen". The authors would like to thank the consortium for the successful cooperation.

show abstract

Section: Applicationsmentioning

confidence: 99%

Section: Applicationsmentioning

confidence: 99%

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

Wörmann¹,

Bogdoll²,

Bührle³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Henson et al [41] presented an ontology-based method for searching scenes in AD datasets. Similar approaches representing the context in a driving scenario are shown in [24,26,38,74]. Ontologies have also been used for context-dependent recommendation tasks [108,40].…”

Section: Knowledge Representation Learningmentioning

confidence: 99%

“…[56,15] Map representation [98] Decision making [87] Context learning [112] Map integration [84] Rules [117,115,116] KG-scene-graphs [114,34] Map updating [85] Reasoning [48,113] KG-based detection [110] Quality of maps [86] Rule learning [19,47,77] Common-sense [17] Scene understanding KG from text [22] Road sign recog. [59,76] Context model [107] Validation Lane detection [72,46] Situation understanding [38] Risk assessm. [9,81,109] Segmentation Behavior Prediction Test gener.…”

Section: Semantic Segmentationmentioning

confidence: 99%

A Survey on Knowledge Graph-based Methods for Automated Driving

Luettin¹,

Monka²,

Henson³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automated driving is one of the most active research areas in computer science. Deep learning methods have made remarkable breakthroughs in machine learning in general and in automated driving (AD) in particular. However, there are still unsolved problems to guarantee reliability and safety of automated systems, especially to effectively incorporate all available information and knowledge in the driving task. Knowledge graphs (KG) have recently gained significant attention from both industry and academia for applications that benefit by exploiting structured, dynamic, and relational data. The complexity of graph-structured data with complex relationships and inter-dependencies between objects has posed significant challenges to existing machine learning algorithms. However, recent progress in knowledge graph embeddings and graph neural networks allows to applying machine learning to graph-structured data. Therefore, we motivate and discuss the potential benefit of KGs applied to the main tasks of AD including 1) ontologies 2) perception, 3) scene understanding, 4) motion planning, and 5) validation. Then, we survey, analyze and categorize ontologies and KG-based approaches for AD. We discuss current research challenges and propose promising future research directions for KG-based solutions for AD.

show abstract

“…A few recent works in the area of AD have also explored the quality of KGEs based on intrinsic evaluation metrics ( Wickramarachchi et al, 2020 ), synthetic data based KGs ( Halilaj et al, 2021 ), and the integration of external knowledge with scene graphs ( Suchan et al, 2020 ).…”

Section: Related Workmentioning

confidence: 99%

Knowledge-infused Learning for Entity Prediction in Driving Scenes

2021

View full text Add to dashboard Cite

Scene understanding is a key technical challenge within the autonomous driving domain. It requires a deep semantic understanding of the entities and relations found within complex physical and social environments that is both accurate and complete. In practice, this can be accomplished by representing entities in a scene and their relations as a knowledge graph (KG). This scene knowledge graph may then be utilized for the task of entity prediction, leading to improved scene understanding. In this paper, we will define and formalize this problem as Knowledge-based Entity Prediction (KEP). KEP aims to improve scene understanding by predicting potentially unrecognized entities by leveraging heterogeneous, high-level semantic knowledge of driving scenes. An innovative neuro-symbolic solution for KEP is presented, based on knowledge-infused learning, which 1) introduces a dataset agnostic ontology to describe driving scenes, 2) uses an expressive, holistic representation of scenes with knowledge graphs, and 3) proposes an effective, non-standard mapping of the KEP problem to the problem of link prediction (LP) using knowledge-graph embeddings (KGE). Using real, complex and high-quality data from urban driving scenes, we demonstrate its effectiveness by showing that the missing entities may be predicted with high precision (0.87 Hits@1) while significantly outperforming the non-semantic/rule-based baselines.

show abstract

A Knowledge Graph-Based Approach for Situation Comprehension in Driving Scenarios

Cited by 22 publications

References 36 publications

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

A Survey on Knowledge Graph-based Methods for Automated Driving

Knowledge-infused Learning for Entity Prediction in Driving Scenes

Contact Info

Product

Resources

About