Neural, symbolic and neural-symbolic reasoning on knowledge graphs

Zhang, Jing; Chen, Bo; Zhang, Lingxi; Ke, Xirui; Ding, Haipeng

doi:10.1016/j.aiopen.2021.03.001

Cited by 50 publications

(25 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are six studies in this category, all but one of which leverage relational structure in some manner. A common theme is the use of graph representations and/or GNNs which aligns with recent research directions proposed by Garcez et al in [21], as well as Zhang et al in [24]. We call Level 3 Cooperative, as it is conceptually similar to Reinforcement Learning (RL).…”

Section: Nesy Categoriesmentioning

confidence: 81%

See 1 more Smart Citation

Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

Hamilton¹,

Nayak²,

Božić³

et al. 2022

Preprint

View full text Add to dashboard Cite

Advocates for Neuro-Symbolic AI (NeSy) assert that combining deep learning with symbolic reasoning will lead to stronger AI than either paradigm on its own. As successful as deep learning has been, it is generally accepted that even our best deep learning systems are not very good at abstract reasoning. And since reasoning is inextricably linked to language, it makes intuitive sense that Natural Language Processing (NLP), would be a particularly well-suited candidate for NeSy. We conduct a structured review of studies implementing NeSy for NLP, challenges and future directions, and aim to answer the question of whether NeSy is indeed meeting its promises: reasoning, out-of-distribution generalization, interpretability, learning and reasoning from small data, and transferability to new domains. We examine the impact of knowledge representation, such as rules and semantic networks, language structure and relational structure, and whether implicit or explicit reasoning contributes to higher promise scores. We find that knowledge encoded in relational structures and explicit reasoning tend to lead to more NeSy goals being satisfied. We also advocate for a more methodical approach to the application of theories of reasoning, which we hope can reduce some of the friction between the symbolic and sub-symbolic schools of AI.

show abstract

Section: Nesy Categoriesmentioning

confidence: 81%

“…Several surveys have already been conducted which cover the overall NeSy landscape going as far back as 2005, and as recently as 2021, so we will not attempt to replicate that here [1,4,8,[13][14][15][16][17][18][19][20][21][22][23][24][25]. In fact, our understanding of the field is guided by the works of these scholars.…”

Section: Contributionsmentioning

confidence: 99%

Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

Hamilton¹,

Nayak²,

Božić³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, these works merely target link prediction, but cannot perform CQA like we do, and extensions to target these more general queries are not straightforward. For other less related neural-symbolic methods we refer the reader to (Zhang et al 2020;Bianchi et al 2020).…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, embedding methods are essentially tailored towards a form of inductive reasoning: given a number of queries and their answers, they are used to predict answers to other similar queries, but they typically ignore ontological knowledge accompanying KGs. Since large portions of expert knowledge can be conveniently represented using ontologies, the benefits of coupling the ontological reasoning and embedding-based methods for KG completion are evident and have been acknowledged in several works (Bianchi et al 2020;Zhang et al 2020;Gutiérrez-Basulto and Schockaert 2018;Kulmanov et al 2019). However, to the best of our knowledge such coupling for CQA has not been studied so far.…”

Section: Introductionmentioning

confidence: 99%

A Neural-symbolic Approach for Ontology-mediated Query Answering

Andresel,

Domokos,

Stepanova

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, low-dimensional vector space representations of knowledge graphs (KGs) have been applied to find answers to conjunctive queries (CQs) over incomplete KGs. However, the current methods only focus on inductive reasoning, i.e. answering CQs by predicting facts based on patterns learned from the data, and lack the ability of deductive reasoning by applying external domain knowledge. Such (expert or commonsense) domain knowledge is an invaluable resource which can be used to advance machine intelligence. To address this shortcoming, we introduce a neural-symbolic method for ontology-mediated CQ answering over incomplete KGs that operates in the embedding space. More specifically, we propose various data augmentation strategies to generate training queries using query-rewriting based methods and then exploit a novel loss function for training the model. The experimental results demonstrate the effectiveness of our training strategies and the new loss function, i.e., our method significantly outperforms the baseline in the settings that require both inductive and deductive reasoning.

show abstract

“…Here, System 2 refers to the System 1/System 2 dual process theory of human reasoning explicated by psychologist and Nobel laureate Daniel Kahneman in his 2011 book "Thinking, Fast and Slow" [77]. AI researchers [6,51,87,96,104,152,164] have drawn many parallels between the characteristics of sub-symbolic and symbolic AI systems and human reasoning with System 1/System 2. Broadly speaking, sub-symbolic (neural, deep-learning) architectures are said to be akin to the fast, intuitive, often biased and/or logically flawed System 1.…”

Section: Introductionmentioning

confidence: 99%

Is neuro-symbolic AI meeting its promises in natural language processing? A structured review

Hamilton

Nayak

Božić

et al. 2024

View full text Add to dashboard Cite

Advocates for Neuro-Symbolic Artificial Intelligence (NeSy) assert that combining deep learning with symbolic reasoning will lead to stronger AI than either paradigm on its own. As successful as deep learning has been, it is generally accepted that even our best deep learning systems are not very good at abstract reasoning. And since reasoning is inextricably linked to language, it makes intuitive sense that Natural Language Processing (NLP), would be a particularly well-suited candidate for NeSy. We conduct a structured review of studies implementing NeSy for NLP, with the aim of answering the question of whether NeSy is indeed meeting its promises: reasoning, out-of-distribution generalization, interpretability, learning and reasoning from small data, and transferability to new domains. We examine the impact of knowledge representation, such as rules and semantic networks, language structure and relational structure, and whether implicit or explicit reasoning contributes to higher promise scores. We find that systems where logic is compiled into the neural network lead to the most NeSy goals being satisfied, while other factors such as knowledge representation, or type of neural architecture do not exhibit a clear correlation with goals being met. We find many discrepancies in how reasoning is defined, specifically in relation to human level reasoning, which impact decisions about model architectures and drive conclusions which are not always consistent across studies. Hence we advocate for a more methodical approach to the application of theories of human reasoning as well as the development of appropriate benchmarks, which we hope can lead to a better understanding of progress in the field. We make our data and code available on github for further analysis.11 https://github.com/kyleiwaniec/neuro-symbolic-ai-systematic-review

show abstract

Neural, symbolic and neural-symbolic reasoning on knowledge graphs

Cited by 50 publications

References 22 publications

Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

A Neural-symbolic Approach for Ontology-mediated Query Answering

Is neuro-symbolic AI meeting its promises in natural language processing? A structured review

Contact Info

Product

Resources

About