EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case

Díaz-Rodríguez, Natalia; Lamas, A.; Sanchez, Jules; Franchi, Gianni; Donadello, Ivan; Tabik, Siham; Filliat, David; Cruz, Policarpo; Montes, Rosana; Herrera, Francisco

doi:10.1016/j.inffus.2021.09.022

Cited by 56 publications

(41 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, self-explainable [12] models perform recommendation and explanation generation jointly. Representative methods of this family include those performing reasoning either on the paths within the KG [19,16,15,11] or via neural symbolic techniques [6,17]. Though the resulting explanation is aligned to the associated recommendation, the utility of the recommendations could decrease and the explanations could not be aligned to the expectation of the users.…”

Section: Introductionmentioning

confidence: 99%

XRecSys: A framework for path reasoning quality in explainable recommendation

et al. 2022

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

XRecSys: A framework for path reasoning quality in explainable recommendation

et al. 2022

View full text Add to dashboard Cite

“…For this reason, a possible future direction would be to design a new post processing stage based on a neural network that draws the line for us, and possibly even go further by filtering with (higher order) axioms from the ontology. Another area of interest for possible future research is integrating existing ontological knowledge directly into the main scene graph generation network, perhaps in the form of a new term in the loss function [14], or through incorporating neurosymbolic propositional and first order logic directly as part of the training process [4].…”

Section: Discussionmentioning

confidence: 99%

“…Another justification for our interest in scene graphs is their potential for usage in neurosymbolic (NeSy) computation approaches. Such approaches are increasingly being used to improve the explainability of AI solutions (XAI) [14]. In this section we detail our survey of the state of the art, as well as what we believe to be their relevance and contribution to solving domain specific problems like ours.…”

Section: Related Workmentioning

confidence: 99%

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Amodeo¹,

Caballero²,

Díaz-Rodríguez³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Scene graph generation from images is a task of great interest to applications such as robotics, because graphs are the main way to represent knowledge about the world and regulate human-robot interactions in tasks such as Visual Question Answering (VQA). Unfortunately, its corresponding area of machine learning is still relatively in its infancy, and the solutions currently offered do not specialize well in concrete usage scenarios. Specifically, they do not take existing "expert" knowledge about the domain world into account; and that might indeed be necessary in order to provide the level of reliability demanded by the use case scenarios. In this paper, we propose an initial approximation to a framework called Ontology-Guided Scene Graph Generation (OG-SGG), that can improve the performance of an existing machine learning based scene graph generator using prior knowledge supplied in the form of an ontology; and we present results evaluated on a specific scenario founded in telepresence robotics.

show abstract

“…Recently there has been a surge in the area of symbolic learning [58]. It represents concepts using symbols and then relationships are defined amongst them.…”

Section: Discussionmentioning

confidence: 99%