Parameter Learning in Probabilistic Databases: A Least Squares Approach

In this paper we propose to apply the Information Bottleneck (IB) approach to the sub-class of Statistical Relational Learning (SRL) languages that are reducible to Bayesian networks. When the resulting networks involve hidden variables, learning these languages requires the use of techniques for learning from incomplete data such as the Expectation Maximization (EM) algorithm. Recently, the IB approach was shown to be able to avoid some of the local maxima in which EM can get trapped when learning with hidden variables. Here we present the algorithm Relational Information Bottleneck (RIB) that learns the parameters of SRL languages reducible to Bayesian Networks. In particular, we present the specialization of RIB to a language belonging to the family of languages based on the distribution semantics, Logic Programs with Annotated Disjunction (LPADs). This language is prototypical for such a family and its equivalent Bayesian networks contain hidden variables. RIB is evaluated on the IMDB, Cora and artificial datasets and compared with LeProbLog, EM, Alchemy and PRISM. The experimental results show that RIB has good performances especially when some logical atoms are unobserved. Moreover, it is particularly suitable when learning from interpretations that share the same Herbrand base.

show abstract

“…| as in Gutmann et al (2008). The other measures are computed using a testing set composed by 10,000 randomly sampled interpretations.…”

Section: Methodsmentioning

confidence: 99%

Applying the information bottleneck to statistical relational learning

Riguzzi

Mauro

2011

Mach Learn

View full text Add to dashboard Cite

show abstract

“…Another, important use of ProbLog is as a vehicle for developing learning and mining algorithms and tools [13,39,19,31], an aspect that we have also discussed in the present paper. In the context of probabilistic representations [49,50], one typically distinguishes two types of learning: parameter estimation and structure learning.…”

Section: Related Work In Statistical Relational Learningmentioning

confidence: 98%

“…Section 10.8. [19] therefore introduce the k-probability P k (q|T ), which approximates the success probability by using the k best (that is, most likely) explanations instead of all proofs when building the DNF formula used in Equation (10.3):…”

Section: K-bestmentioning

confidence: 99%

“…The parameter estimation technique should then determine the best values for the parameters. Our approach as implemented in LeProbLog [19,46] (Least Square Parameter Estimation for ProbLog) performs a gradient-based search to minimize the error on the given training data. The problem tackled can be formalized as regression task as follows:…”

Section: Parameter Estimationmentioning

confidence: 99%

“…Therefore, LeProbLog can use examples of both forms, (atomic) queries and proofs, at the same time. For further details and experimental results in the context of the biological network application, we refer to [19,46].…”

Section: Example 101 (Gradient Of a Query)mentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic Inductive Querying Using ProbLog

Raedt

Kimmig

Gutmann

et al. 2010

Inductive Databases and Constraint-Based Data Mining

Self Cite

View full text Add to dashboard Cite

We study how probabilistic reasoning and inductive querying can be combined within ProbLog, a recent probabilistic extension of Prolog. ProbLog can be regarded as a database system that supports both probabilistic and inductive reasoning through a variety of querying mechanisms. After a short introduction to ProbLog, we provide a survey of the different types of inductive queries that ProbLog supports, and show how it can be applied to the mining of large biological networks. IntroductionIn recent years, both probabilistic and inductive databases have received considerable attention in the literature. Probabilistic databases [1] allow one to represent and reason about uncertain data, while inductive databases [2] aim at tight integration of data mining primitives in database query languages. Despite the current interest in these types of databases, there have, to the best of the authors' knowledge, been no attempts to integrate these two trends of research. This chapter wants to contribute to a better understanding of the issues involved by providing a survey of the developments around ProbLog [3] 1 , an extension of Prolog, which supports both inductive and probabilistic querying. ProbLog has been motivated by the need to develop intelligent tools for supporting life scientists analyzing large biological networks. The analysis of such networks typically involves uncertain data, requiring probabilistic representations and inference, as well as the need to find patterns in data, and hence, supporting data mining. ProbLog can be conveniently regarded as a probabilistic database supporting several types of inductive and probabilistic queries. This paper provides an overview of the different types of queries that ProbLog supports.A ProbLog program defines a probability distribution over logic programs (or databases) by specifying for each fact (or tuple) the probability that it belongs to a randomly sampled program (or database), where probabilities are mutually independent. The semantics of ProbLog is then defined by the success probability of a query, which corresponds to the probability that the query succeeds in a randomly sampled program (or database). ProbLog is closely related to other probabilistic logics and probabilistic databases that have been developed over the past two decades to face the general need of combining deductive abilities with reasoning about uncertainty, see e.g. [4,5,6,7,8]. The semantics of ProbLog is studied in Section 10.2. In Section 10.10, we discuss related work in statistical relational learning.We now give a first overview of the types of queries ProbLog supports. Throughout the chapter, we use the graph in Figure 1(a) for illustration, inspired on the application in biological networks discussed in Section 10.9. It contains several nodes (representing entities) as well as edges (representing relationships). Furthermore, the edges are probabilistic, that is, they are present only with the probability indicated. Probabilistic Inference What is the probability that a query...

show abstract

A Constrained Optimization Approach to Set the Parameters of Probabilistic Answer Set Programs

Azzolini

2023

Inductive Logic Programming

View full text Add to dashboard Cite

Parameter Learning in Probabilistic Databases: A Least Squares Approach

Cited by 42 publications

References 16 publications

Applying the information bottleneck to statistical relational learning

Applying the information bottleneck to statistical relational learning

Probabilistic Inductive Querying Using ProbLog

A Constrained Optimization Approach to Set the Parameters of Probabilistic Answer Set Programs

Contact Info

Product

Resources

About