Logic Tensor Networks

Badreddine, Samy; Garcez, Artur d’Avila; Serafini, Luciano; Spranger, Michael

doi:10.1016/j.artint.2021.103649

Cited by 70 publications

(60 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this reason, a possible future direction would be to design a new post processing stage based on a neural network that draws the line for us, and possibly even go further by filtering with (higher order) axioms from the ontology. Another area of interest for possible future research is integrating existing ontological knowledge directly into the main scene graph generation network, perhaps in the form of a new term in the loss function [14], or through incorporating neurosymbolic propositional and first order logic directly as part of the training process [4].…”

Section: Discussionmentioning

confidence: 99%

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Amodeo¹,

Caballero²,

Díaz-Rodríguez³

et al. 2022

Preprint

View full text Add to dashboard Cite

Scene graph generation from images is a task of great interest to applications such as robotics, because graphs are the main way to represent knowledge about the world and regulate human-robot interactions in tasks such as Visual Question Answering (VQA). Unfortunately, its corresponding area of machine learning is still relatively in its infancy, and the solutions currently offered do not specialize well in concrete usage scenarios. Specifically, they do not take existing "expert" knowledge about the domain world into account; and that might indeed be necessary in order to provide the level of reliability demanded by the use case scenarios. In this paper, we propose an initial approximation to a framework called Ontology-Guided Scene Graph Generation (OG-SGG), that can improve the performance of an existing machine learning based scene graph generator using prior knowledge supplied in the form of an ontology; and we present results evaluated on a specific scenario founded in telepresence robotics.

show abstract

Section: Discussionmentioning

confidence: 99%

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Amodeo¹,

Caballero²,

Díaz-Rodríguez³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Another aspect related to scalability is the choice of aggregation function and fuzzy logic operators. Emilie van Krieken et al [14] and Samy Badreddine [4] found substantial differences between differential fuzzy logic operators in terms of computational efficiency, scalability, gradients, and ability to handle exceptions, which are important characteristics in a learning setting. Their analysis lays the groundwork for the present FasterLTN architecture, which incorporates and extends the log-product aggregator analyzed in [14].…”

Section: Related Workmentioning

confidence: 99%

“…We first summarize the Faster R-CNN overall architecture (Section 3.1). Then, we introduce the main concepts behind LTNs (Section 3.2) and their application to object detection (Section 3.3), referring the reader to [3,4] for additional details. Finally, the joint training procedure of Faster-LTN is explained in Section 3.4, highlighting the main changes introduced to make end-to-end training feasible.…”

Section: The Faster-ltn Architecturementioning

confidence: 99%

“…The present work leverages the Logic Tensor Network (LTN) paradigm that was proposed by Serafini, Donadello and d'Avila Garcez [3,4]. In very simple terms, LTNs operate by interpreting (or grounding) a First-Order Logic (FOL) as functions on real vectors, which parameters can be trained via stochastic gradient descents to maximize the satisfiability of a given theory.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Faster-LTN: a neuro-symbolic, end-to-end object detection architecture

Manigrasso,

Miro,

Morra

et al. 2021

Preprint

View full text Add to dashboard Cite

The detection of semantic relationships between objects represented in an image is one of the fundamental challenges in image interpretation. Neural-Symbolic techniques, such as Logic Tensor Networks (LTNs), allow the combination of semantic knowledge representation and reasoning with the ability to efficiently learn from examples typical of neural networks. We here propose Faster-LTN, an object detector composed of a convolutional backbone and an LTN. To the best of our knowledge, this is the first attempt to combine both frameworks in an end-to-end training setting. This architecture is trained by optimizing a grounded theory which combines labelled examples with prior knowledge, in the form of logical axioms. Experimental comparisons show competitive performance with respect to the traditional Faster R-CNN architecture.

show abstract

“…There has been other work in differentiable neuro-symbolic systems outside of TPRs and VSAs, most notably using first-order logic [20][21][22]. These works represent another powerful alternative approach, though at this time are often more involved in their design and training procedures.…”

Section: Related Workmentioning

confidence: 99%

Learning with Holographic Reduced Representations

Ganesan¹,

Gao²,

Gandhi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Holographic Reduced Representations (HRR) are a method for performing symbolic AI on top of real-valued vectors [1] by associating each vector with an abstract concept, and providing mathematical operations to manipulate vectors as if they were classic symbolic objects. This method has seen little use outside of older symbolic AI work and cognitive science. Our goal is to revisit this approach to understand if it is viable for enabling a hybrid neural-symbolic approach to learning as a differentiable component of a deep learning architecture. HRRs today are not effective in a differentiable solution due to numerical instability, a problem we solve by introducing a projection step that forces the vectors to exist in a well behaved point in space. In doing so we improve the concept retrieval efficacy of HRRs by over 100×. Using multi-label classification we demonstrate how to leverage the symbolic HRR properties to develop an output layer and loss function that is able to learn effectively, and allows us to investigate some of the pros and cons of an HRR neuro-symbolic learning approach.

show abstract

Logic Tensor Networks

Cited by 70 publications

References 51 publications

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Faster-LTN: a neuro-symbolic, end-to-end object detection architecture

Learning with Holographic Reduced Representations

Contact Info

Product

Resources

About