Combining statistical and relational methods for learning in hypertext domains

Slattery, Seán; Craven, Mark

doi:10.1007/bfb0027309

Cited by 50 publications

(26 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative way is to use relational learning [27] methods such as Markov Random Fields [12,11,1], Inductive Logic Programming [43,19], Probabilistic Relational Models [21], Relational Markov Networks [45], Dependency Networks [32] and Lazy Associative Classification [46]. Some comprehensive studies in this area can be found in [27,31].…”

Section: Related Workmentioning

confidence: 99%

Classifying networked entities with modularity kernels

Zhang

Mao

2008

Proceedings of the 17th ACM Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Statistical machine learning techniques for data classification usually assume that all entities are i.i.d. (independent and identically distributed). However, real-world entities often interconnect with each other through explicit or implicit relationships to form a complex network. Although some graph-based classification methods have emerged in recent years, they are not really suitable for complex networks as they do not take the degree distribution of network into consideration. In this paper, we propose a new technique, Modularity Kernel, that can effectively exploit the latent community structure of networked entities for their classification. A number of experiments on hypertext datasets show that our proposed approach leads to excellent classification performance in comparison with the state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Classifying networked entities with modularity kernels

Zhang

Mao

2008

Proceedings of the 17th ACM Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…Another example is collective web-page classification. Given a set of web-pages of a department, the task is to simultaneously classify these web-pages into some pre-defined categories based on their content and the hyperlinks between them (Slattery & Craven, 1998). It is, therefore, an important research problem to develop discriminative learning algorithms for MLNs that improves its predictive performance on these discriminative tasks.…”

Section: Performing Organization Name(s) and Address(es)mentioning

confidence: 99%

Discriminative Learning with Markov Logic Networks

Huynh¹

2009

View full text Add to dashboard Cite

Statistical relational learning (SRL) is an emerging area of research that addresses the problem of learning from noisy structured/relational data. Markov logic networks (MLNs), sets of weighted clauses, are a simple but powerful SRL formalism that combines the expressivity of first-order logic with the flexibility of probabilistic reasoning. Most of the existing learning algorithms for MLNs are in the generative setting: they try to learn a model that maximizes the likelihood of the training data. However, most of the learning problems in relational data are discriminative. So to utilize the power of MLNs, we need discriminative learning methods that well match these discriminative tasks.In this proposal, we present two new discriminative learning algorithms for MLNs. The first one is a discriminative structure and weight learner for MLNs with non-recursive clauses. We use a variant of ALEPH, an off-the-shelf Inductive Logic Programming (ILP) system, to learn a large set of Horn clauses from the training data, then we apply an L 1 -regularization weight learner to select a small set of non-zero weight clauses that maximizes the conditional log-likelihood (CLL) of the training data. The experimental results show that our proposed algorithm outperforms existing learning methods for MLNs and traditional ILP systems in term of predictive accuracy, and its performance is comparable to stateof-the-art results on some ILP benchmarks. The second algorithm we present is a max-margin weight learner for MLNs. Instead of maximizing the CLL of the data like all existing discriminative weight learners for MLNs, the new weight learner tries to maximize the ratio between the probability of the correct label (the observable data) and and the closest incorrect label (among all the wrong labels, this one has the highest probability), which can be formulated as an optimization problem called "1-slack" structural SVM. This optimization problem can be solved by an efficient algorithm based on the cutting plane method. However, this cutting plane algorithm requires an efficient inference method as a subroutine. Unfortunately, exact inference in MLNs is intractable. So we develop a new approximation inference method for MLNs based on Linear Programming relaxation. Extensive experiments in two real-world MLN applications demonstrate that the proposed max-margin weight learner generally achieves higher F 1 scores than the current best discriminative weight learner for MLNs.For future work, our short-term goal is to develop a more efficient inference algorithm and test our max-margin weight learner on more complex problems where there are complicated relationships between the input and output variables and among the outputs. In the longer-term, our plan is to develop more efficient learning algorithms through online learning and algorithms that revise both the clauses and their weights to improve predictive performance. Report Documentation PageForm Public reporting burden for the collection of information is estimated to averag...

show abstract

“…The work by Slattery and Craven (1998) goes in the other direction: it uses naive Bayes to invent new predicates to be used by Foil. The invented predicates are Boolean features of the individual.…”

Section: Related Workmentioning

confidence: 99%

Naive Bayesian Classification of Structured Data

2004

View full text Add to dashboard Cite

Abstract. In this paper we present 1BC and 1BC2, two systems that perform naive Bayesian classification of structured individuals. The approach of 1BC is to project the individuals along first-order features. These features are built from the individual using structural predicates referring to related objects (e.g., atoms within molecules), and properties applying to the individual or one or several of its related objects (e.g., a bond between two atoms). We describe an individual in terms of elementary features consisting of zero or more structural predicates and one property; these features are treated as conditionally independent in the spirit of the naive Bayes assumption. 1BC2 represents an alternative first-order upgrade to the naive Bayesian classifier by considering probability distributions over structured objects (e.g., a molecule as a set of atoms), and estimating those distributions from the probabilities of its elements (which are assumed to be independent). We present a unifying view on both systems in which 1BC works in language space, and 1BC2 works in individual space. We also present a new, efficient recursive algorithm improving upon the original propositionalisation approach of 1BC. Both systems have been implemented in the context of the first-order descriptive learner Tertius, and we investigate the differences between the two systems both in computational terms and on artificially generated data. Finally, we describe a range of experiments on ILP benchmark data sets demonstrating the viability of our approach.

show abstract

Combining statistical and relational methods for learning in hypertext domains

Cited by 50 publications

References 4 publications

Classifying networked entities with modularity kernels

Classifying networked entities with modularity kernels

Discriminative Learning with Markov Logic Networks

Naive Bayesian Classification of Structured Data

Contact Info

Product

Resources

About