Exploring optimization of semantic relationship graph for multi-relational Bayesian classification

Man

2012

Mach Learn

Recently, there has been an increasing interest in generative models that represent probabilistic patterns over both links and attributes. A common characteristic of relational data is that the value of a predicate often depends on values of the same predicate for related entities. For directed graphical models, such recursive dependencies lead to cycles, which violates the acyclicity constraint of Bayes nets. In this paper we present a new approach to learning directed relational models which utilizes two key concepts: a pseudo likelihood measure that is well defined for recursive dependencies, and the notion of stratification from logic programming. An issue for modelling recursive dependencies with Bayes nets are redundant edges that increase the complexity of learning. We propose a new normal form format that removes the redundancy, and prove that assuming stratification, the normal form constraints involve no loss of modelling power. Empirical evaluation compares our approach to learning recursive dependencies with undirected models (Markov Logic Networks). The Bayes net approach is orders of magnitude faster, and learns more recursive dependencies, which lead to more accurate predictions.

Section: Results Neither Of the Markov Logic Methods Lhl Nor Lsm Discmentioning

confidence: 99%

mentioning

confidence: 99%

Learning directed relational models with recursive dependencies

Man

2012

Mach Learn

“…To indicate this dependence, we add the functor node RA(S , P ) as a parent of popularity(P ) in the decision tree for ranking(S ). Like most statistical-relational systems [1,33], and like the learn-andjoin algorithm, we consider associations between attributes of two entities only conditional of the existence of a link between the entities. Therefore there is no branch corresponding to RA(S , P ) = F in the decision tree of Figure 7(right).…”

Section: !"#$%And'!() *"#$%And'*() +$#-#'!() -#/And00-and'!() 23240$+-/5'*()mentioning

confidence: 99%

Learning Compact Markov Logic Networks with Decision Trees

Inductive Logic Programming

Gao

2012

Statistical-relational learning combines logical syntax with probabilistic methods. Markov Logic Networks (MLNs) are a prominent model class that generalizes both first-order logic and undirected graphical models (Markov networks). The qualitative component of an MLN is a set of clauses and the quantitative component is a set of clause weights. Generative MLNs model the joint distribution of relationships and attributes. A state-of-the-art structure learning method is the moralization approach: learn a set of directed Horn clauses, then convert them to conjunctions to obtain MLN clauses. The directed clauses are learned using Bayes net methods. The moralization approach takes advantage of the high-quality inference algorithms for MLNs and their ability to handle cyclic dependencies. A weakness of moralization is that it leads to an unnecessarily large number of clauses. In this paper we show that using decision trees to represent conditional probabilities in the Bayes net is an effective remedy that leads to much more compact MLN structures. In experiments on benchmark datasets, the decision trees reduce the number of clauses in the moralized MLN by a factor of 5-25, depending on the dataset. The accuracy of predictions is competitive with the models obtained by standard moralization, and in many cases superior.

“…To indicate this dependence, we add the functor node RA(S, P ) as a parent of popularity(P ) in the decision tree for ranking (S). Like most statistical-relational systems (Getoor et al 2007;Chen et al 2009), and like the learn-and-join algorithm, we consider associations between attributes of two entities only conditional of the existence of a link between the entities. Therefore there is no branch corresponding to RA(S, P ) = F in the decision tree of Fig.…”

Section: Form a Family Join Table That Combines All Functor Nodes In mentioning

confidence: 99%

Learning compact Markov logic networks with decision trees

Gao

2012

Mach Learn

Statistical-relational learning combines logical syntax with probabilistic methods. Markov Logic Networks (MLNs) are a prominent model class that generalizes both firstorder logic and undirected graphical models (Markov networks). The qualitative component of an MLN is a set of clauses and the quantitative component is a set of clause weights. Generative MLNs model the joint distribution of relationships and attributes. A state-ofthe-art structure learning method is the moralization approach: learn a set of directed Horn clauses, then convert them to conjunctions to obtain MLN clauses. The directed clauses are learned using Bayes net methods. The moralization approach takes advantage of the high-quality inference algorithms for MLNs and their ability to handle cyclic dependencies. A weakness of moralization is that it leads to an unnecessarily large number of clauses. In this paper we show that using decision trees to represent conditional probabilities in the Bayes net is an effective remedy that leads to much more compact MLN structures. In experiments on benchmark datasets, the decision trees reduce the number of clauses in the moralized MLN by a factor of 5-25, depending on the dataset. The accuracy of predictions is competitive with the models obtained by standard moralization, and in many cases superior.