Approximate Inference for Neural Probabilistic Logic Programming

Manhaeve, Robin; Marra, Giuseppe; Raedt, Luc De

doi:10.24963/kr.2021/45

Cited by 4 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approximate knowledge compilation is the field of research that deals with tackling this issue. While it contains interesting recent work [Fierens et al, 2015, Huang et al, 2021, Manhaeve et al, 2021b, it was highlighted by Manhaeve et al that the introduction of the neural paradigm does lead to further complications. As such, we opted for exact knowledge compilation, but it has to be noted that we will be able to benefit from any future advances in the field of approximate inference.…”

Section: H Limitationsmentioning

confidence: 99%

Neural Probabilistic Logic Programming in Discrete-Continuous Domains

Smet¹,

Martires²,

Manhaeve³

et al. 2023

Preprint

View full text Add to dashboard Cite

Neural-symbolic AI (NeSy) allows neural networks to exploit symbolic background knowledge in the form of logic. It has been shown to aid learning in the limited data regime and to facilitate inference on out-of-distribution data. Probabilistic NeSy focuses on integrating neural networks with both logic and probability theory, which additionally allows learning under uncertainty. A major limitation of current probabilistic NeSy systems, such as DeepProbLog, is their restriction to finite probability distributions, i.e., discrete random variables. In contrast, deep probabilistic programming (DPP) excels in modelling and optimising continuous probability distributions. Hence, we introduce DeepSeaProbLog, a neural probabilistic logic programming language that incorporates DPP techniques into NeSy. Doing so results in the support of inference and learning of both discrete and continuous probability distributions under logical constraints. Our main contributions are 1) the semantics of DeepSeaProbLog and its corresponding inference algorithm, 2) a proven asymptotically unbiased learning algorithm, and 3) a series of experiments that illustrate the versatility of our approach.

show abstract

Section: H Limitationsmentioning

confidence: 99%

Neural Probabilistic Logic Programming in Discrete-Continuous Domains

Smet¹,

Martires²,

Manhaeve³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We study three Neurosymbolic reasoning tasks to evaluate the performance and scalability of A-NeSI: Multi-digit MNISTAdd [139], Visual Sudoku Puzzle Classification [10] and Warcraft path planning. Code is available at https://github.com/HEmile/a-nesi.…”

Section: Methodsmentioning

confidence: 99%

“…y i and w i are one-hot encoded digits, except for the first output digit y 1 : it can only be 0 or 1. We used a shared set of hyperparameters for all N. Like [138,139], we take the MNIST [122] dataset and use each digit exactly once to create data. We follow [138] and require more unique digits for increasing N. Therefore, the training dataset will be of size 60000/2N and the test dataset of size 10000/2N.…”

Section: Multi-digit Mnistaddmentioning

confidence: 99%

“…Other neurosymbolic methods use fuzzy logics [12,55,45,73], which are faster than PNL with exact inference. Although traversing ground formulas is linear time, the grounding is itself often exponential [139], so the scalability of fuzzy logics often fails to deliver. A-NESI is polynomial in the number of ground atoms and does not traverse the ground formula.…”

Section: Related Workmentioning

confidence: 99%

“…A further study into this t-norm may result in a refinement function inheriting the beneficial properties of product fuzzy logic as recommended in Chapter 3. Finally, while we have initial experiments on the effectiveness of ILR in the popular MNIST Addition [139] benchmark, there is little empirical work on the effectiveness of the ILR layer in practical neurosymbolic settings.…”

Section: Research Question 2: Iterative Local Refinementmentioning

confidence: 99%

See 2 more Smart Citations

Optimisation in Neurosymbolic Learning Systems

van Krieken

View full text Add to dashboard Cite

In the last few years, Artificial Intelligence (AI) has reached the public consciousness through high-profile applications such as chatbots, image generators, speech synthesis and transcription. These are all due to the success of deep learning: Machine learning algorithms that learn tasks from massive amounts of data. The neural network models used in deep learning involve many parameters, often in the order of billions. These models often fail on tasks that computers are traditionally very good at, like calculating arithmetic expressions, reasoning about many different pieces of information, planning and scheduling complex systems, and retrieving information from a database. These tasks are traditionally solved using symbolic methods in AI based on logic and formal reasoning. Neurosymbolic AI instead aims to integrate deep learning with symbolic AI. This integration has many promises, such as decreasing the amount of data required to train the neural networks, improving the explainability and interpretability of answers given by models and verifying the correctness of trained systems. We mainly study neurosymbolic learning, where we have, in addition to data, background knowledge expressed using symbolic languages. How do we connect the symbolic and neural components to communicate this knowledge to the neural networks? We consider two answers: Fuzzy and probabilistic reasoning. Fuzzy reasoning studies degrees of truth. A person can be very or somewhat tall: Tallness is not a binary concept. Instead, probabilistic reasoning studies the probability that something is true or will happen. A coin has a 0.5 probability of landing heads. We never say it landed on "somewhat heads". What happens when we use fuzzy (part I) or probabilistic (part II) approaches to neurosymbolic learning? Moreover, do these approaches use the background knowledge we expect them to? Our first research question studies how different forms of fuzzy reasoning combine with learning. We find surprising results like a connection to the Raven paradox, which states that we confirm "ravens are black" when we observe a green apple. In this study, we gave our neural network a training objective created from the background knowledge. However, we did not use the background knowledge when we deployed our models after training. In our second research question, we studied how to use background knowledge in deployed models. To this end, we developed a new neural network layer based on fuzzy reasoning. The remaining research questions study probabilistic approaches to neurosymbolic learning. Probabilistic reasoning is a natural fit for neural networks, which we usually train to be probabilistic. However, probabilistic approaches come at a cost: They are expensive to compute and do not scale well to large tasks. In our third research question, we study how to connect probabilistic reasoning with neural networks by sampling to estimate averages. Sampling circumvents computing reasoning outcomes for all input combinations. In the fourth and final research question, we study scaling probabilistic neurosymbolic learning to much larger problems than possible before. Our insight is to train a neural network to predict the result of probabilistic reasoning. We perform this training process with just the background knowledge: We do not collect data. How is this related to optimisation? All research questions are related to optimisation problems. Within neurosymbolic learning, optimisation with popular methods like gradient descent undertake a form of reasoning. There is ample opportunity to study how this optimisation perspective improves our neurosymbolic learning methods. We hope this dissertation provides some of the answers needed to make practical neurosymbolic learning a reality: Where practitioners provide both data and knowledge that the neurosymbolic learning methods use as efficiently as possible to train the next generation of neural networks.

show abstract