On Discriminative Bayesian Network Classifiers and Logistic Regression

Roos, Teemu; Wettig, Hannes; Grünwald, Peter; Myllymäki, Petri; Tirri, Henry

doi:10.1007/s10994-005-0471-6

Cited by 70 publications

(62 citation statements)

References 18 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead of optimizing the loglikelihood, discriminative approaches try to maximize the conditional likelihood and may produce better class probability estimates [14], [24], [3]. A discriminant version of SNODE can be developed by maximizing the conditional likelihood; however, the corresponding optimization problem may be more difficult than the current one, since the conditional likelihood does not decompose into separate terms for each function to be learned.…”

Section: Discussionmentioning

confidence: 99%

Semi-naive Exploitation of One-Dependence Estimators

Zhou

2009

2009 Ninth IEEE International Conference on Data Mining

View full text Add to dashboard Cite

Abstract-It is well known that the key of Bayesian classifier learning is to balance the two important issues, that is, the exploration of attribute dependencies in high orders for ensuring a sufficient flexibility in approximating the ground-truth dependencies, and the exploration of low orders for ensuring a stable probability estimate from limited training samples. By allowing one-order attribute dependencies, one-dependence estimators (ODEs) have been shown to be able to approximate the ground-truth attribute dependencies whilst keeping the effectiveness of probability estimation, and therefore leading to excellent performance. In previous studies, however, ODEs were exploited in simple ways, such as by averaging, for classification. In this paper, we propose a semi-naive exploitation of ODEs that fits a function of ODEs to pursue higher-order attribute dependencies. Extensive experiments show that the proposed SNODE approach can achieve better performance than many state-of-the-art Bayesian classifiers.

show abstract

Section: Discussionmentioning

confidence: 99%

Semi-naive Exploitation of One-Dependence Estimators

Zhou

2009

2009 Ninth IEEE International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…A second contribution of this work is the development of a simple framework to characterize the parameter learning task for Bayesian network classifiers. Building on previous work by Friedman et al (1997), Greiner et al (2005), Pernkopf and Wohlmayr (2009), Roos et al (2005) and Zaidi et al (2013), this framework allows us to lay out the different techniques in a systematic manner; highlighting similarities, distinctions and equivalences.…”

Section: Introductionmentioning

confidence: 94%

Efficient parameter learning of Bayesian network classifiers

et al. 2017

View full text Add to dashboard Cite

Recent advances have demonstrated substantial benefits from learning with both generative and discriminative parameters. On the one hand, generative approaches address the estimation of the parameters of the joint distribution-P(y, x), which for most network types is very computationally efficient (a notable exception to this are Markov networks) and on the other hand, discriminative approaches address the estimation of the parameters of the posterior distribution-and, are more effective for classification, since they fit P(y|x) directly. However, discriminative approaches are less computationally efficient as the normalization factor in the conditional log-likelihood precludes the derivation of closed-form estimation of parameters. This paper introduces a new discriminative parameter learning method for Bayesian network classifiers that combines in an elegant fashion parameters learned using Editors: Thomas Gärtner, Mirco Nanni, Andrea Passerini, and Celine Robardet. Learn (2017Learn ( ) 106:1289Learn ( -1329 both generative and discriminative methods. The proposed method is discriminative in nature, but uses estimates of generative probabilities to speed-up the optimization process. A second contribution is to propose a simple framework to characterize the parameter learning task for Bayesian network classifiers. We conduct an extensive set of experiments on 72 standard datasets and demonstrate that our proposed discriminative parameterization provides an efficient alternative to other state-of-the-art parameterizations.

show abstract

“…It is widely recognized that it is generally too hard to perform both general structure search and optimization of discriminative parameter values. Still, a limited amount of structure selection is performed even with discriminative parameter values, such as step-wise model selection [21] or TAN-like structures [3,20], but rarely one can afford an exhaustive search for interactions.…”

Section: Empirical Evaluationmentioning

confidence: 99%

Bayesian Learning of Markov Network Structure

Jakulin

Rish²

2006

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We propose a simple and efficient approach to building undirected probabilistic classification models (Markov networks) that extend naïve Bayes classifiers and outperform existing directed probabilistic classifiers (Bayesian networks) of similar complexity. Our Markov network model is represented as a set of consistent probability distributions on subsets of variables. Inference with such a model can be done efficiently in closed form for problems like class probability estimation. We also propose a highly efficient Bayesian structure learning algorithm for conditional prediction problems, based on integrating along a hill-climb in the structure space. Our prior based on the degrees of freedom effectively prevents overfitting.

show abstract

On Discriminative Bayesian Network Classifiers and Logistic Regression

Cited by 70 publications

References 18 publications

Semi-naive Exploitation of One-Dependence Estimators

Semi-naive Exploitation of One-Dependence Estimators

Efficient parameter learning of Bayesian network classifiers

Bayesian Learning of Markov Network Structure

Contact Info

Product

Resources

About