Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints

Chu, Jielei; Wang, Hongjun; Meng, Hua; Jin, Peng; Li, Tianrui

doi:10.48550/arxiv.1701.03647

Cited by 1 publication

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The simplest approach trains a BM by interpreting continuous data as the expectations needed to compute the gradients of the RBM. Other more mathematically justified approaches involve fully continuous models such as the harmonium [38,39], and continuous-discrete models such as RBMs with Gaussian visible units and discrete latent units [26].…”

Section: Gaussian Smoothing Of Boltzmann Machinesmentioning

confidence: 99%

See 1 more Smart Citation

RBM-Flow and D-Flow: Invertible Flows with Discrete Energy Base Spaces

O’Connor¹,

Vinci²

2020

Preprint

View full text Add to dashboard Cite

Efficient sampling of complex data distributions can be achieved using trained invertible flows (IF), where the model distribution is generated by pushing a simple base distribution through multiple nonlinear bijective transformations. However, the iterative nature of the transformations in IFs can limit the approximation to the target distribution. In this paper we seek to mitigate this by implementing RBM-Flow, an IF model whose base distribution is a Restricted Boltzmann Machine (RBM) with a continuous smoothing applied. We show that by using RBM-Flow we are able to improve the quality of samples generated, quantified by the Inception Scores (IS) and Frechet Inception Distance (FID), over baseline models with the same IF transformations, but with less expressive base distributions. Furthermore, we also obtain D-Flow, an IF model with uncorrelated discrete latent variables. We show that D-Flow achieves similar likelihoods and FID/IS scores to those of a typical IF with Gaussian base variables, but with the additional benefit that global features are meaningfully encoded as discrete labels in the latent space.

show abstract

Section: Gaussian Smoothing Of Boltzmann Machinesmentioning

confidence: 99%

“…Specifically, we look at using an RBM as our discrete EBM due to the efficient mixing we can get from using PCD with block Gibbs sampling. There have been several attempts in the literature to smooth RBMs to model continuous variables [25,26]. We found the most effective technique to be the Gaussian smoothing introduced early on in Ref.…”

Section: Introductionmentioning

confidence: 99%

RBM-Flow and D-Flow: Invertible Flows with Discrete Energy Base Spaces

O’Connor¹,

Vinci²

2020

Preprint

View full text Add to dashboard Cite

show abstract

Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints

Cited by 1 publication

References 45 publications

RBM-Flow and D-Flow: Invertible Flows with Discrete Energy Base Spaces

RBM-Flow and D-Flow: Invertible Flows with Discrete Energy Base Spaces

Contact Info

Product

Resources

About