SPFlow: An Easy and Extensible Library for Deep Probabilistic Learning using Sum-Product Networks

Molina, Alejandro; Vergari, Antonio; Stelzner, Karl; Peharz, Robert; Subramani, Pranav; Mauro, Nicola Di; Poupart, Pascal; Kersting, Kristian

doi:10.48550/arxiv.1901.03704

Cited by 8 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We take Bayesian network implementation (Scutari, 2009) and sum-product network (SPN) package (Molina et al, 2019;Poon and Domingos, 2011) as experimental baselines. 4 500 samples are used for training, and the rest 500 samples are for testing.…”

Section: Comparison With Graphical Modelsmentioning

confidence: 99%

Variational Flow Graphical Model

Ren¹,

Karimi²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper introduces a novel approach to embed flow-based models with hierarchical structures. The proposed framework is named Variational Flow Graphical (VFG) Model. VFGs learn the representation of high dimensional data via a message-passing scheme by integrating flowbased functions through variational inference. By leveraging the expressive power of neural networks, VFGs produce a representation of the data using a lower dimension, thus overcoming the drawbacks of many flow-based models, usually requiring a high dimensional latent space involving many trivial variables. Aggregation nodes are introduced in the VFG models to integrate forward-backward hierarchical information via a message passing scheme. Maximizing the evidence lower bound (ELBO) of data likelihood aligns the forward and backward messages in each aggregation node achieving a consistency node state. Algorithms have been developed to learn model parameters through gradient updating regarding the ELBO objective.

show abstract

Section: Comparison With Graphical Modelsmentioning

confidence: 99%

Variational Flow Graphical Model

Ren¹,

Karimi²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…1). These sub-routines can be easily implemented in circuit libraries such as Juice (Dang et al, 2021) or SPFlow (Molina et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queries

Vergari,

Choi,

Liu

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models. In this paper, we show how complex inference scenarios for these models that commonly arise in machine learning-from computing the expectations of decision tree ensembles to information-theoretic divergences of deep mixture models-can be represented in terms of tractable modular operations over circuits. Specifically, we characterize the tractability of a vocabulary of simple transformations-sums, products, quotients, powers, logarithms, and exponentials-in terms of sufficient structural constraints of the circuits they operate on, and present novel hardness results for the cases in which these properties are not satisfied. Building on these operations, we derive a unified framework for reasoning about tractable models that generalizes several results in the literature and opens up novel tractable inference scenarios.

show abstract

“…At the same time, PCs are analogous to NNs since their evaluation is also carried out using computation graphs. By exploiting the parallel computation power of GPUs, dedicated implementations (Dang et al, 2021;Molina et al, 2019) can train a complex PC with millions of parameters in minutes. These innovations have made PCs much more expressive and scalable to richer datasets that are beyond the reach of "older" TPMs (Peharz et al, 2020a).…”

Section: Introductionmentioning

confidence: 99%

Tractable Regularization of Probabilistic Circuits

Liu,

Broeck

2021

Preprint

View full text Add to dashboard Cite

Probabilistic Circuits (PCs) are a promising avenue for probabilistic modeling. They combine advantages of probabilistic graphical models (PGMs) with those of neural networks (NNs). Crucially, however, they are tractable probabilistic models, supporting efficient and exact computation of many probabilistic inference queries, such as marginals and MAP. Further, since PCs are structured computation graphs, they can take advantage of deep-learning-style parameter updates, which greatly improves their scalability. However, this innovation also makes PCs prone to overfitting, which has been observed in many standard benchmarks. Despite the existence of abundant regularization techniques for both PGMs and NNs, they are not effective enough when applied to PCs. Instead, we re-think regularization for PCs and propose two intuitive techniques, data softening and entropy regularization, that both take advantage of PCs' tractability and still have an efficient implementation as a computation graph. Specifically, data softening provides a principled way to add uncertainty in datasets in closed form, which implicitly regularizes PC parameters. To learn parameters from a softened dataset, PCs only need linear time by virtue of their tractability. In entropy regularization, the exact entropy of the distribution encoded by a PC can be regularized directly, which is again infeasible for most other density estimation models. We show that both methods consistently improve the generalization performance of a wide variety of PCs. Moreover, when paired with a simple PC structure, we achieved state-of-the-art results on 10 out of 20 standard discrete density estimation benchmarks.

show abstract

SPFlow: An Easy and Extensible Library for Deep Probabilistic Learning using Sum-Product Networks

Cited by 8 publications

References 0 publications

Variational Flow Graphical Model

Variational Flow Graphical Model

A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queries

Tractable Regularization of Probabilistic Circuits

Contact Info

Product

Resources

About