Weighted contrastive divergence

Romero, Enrique; Mazzanti, F.; Delgado, Jordi; Buchaca, David

doi:10.1016/j.neunet.2018.09.013

Cited by 15 publications

(10 citation statements)

References 21 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[7], the authors study, in a systematic way, the convergence properties of CD, PCD and PT on several small toy models that can be analyzed exactly, that is, where the LL can be computed by brute-force. We can find more recent works [32,33,34,35] improving the learning scheme for RBMs and yet, still not giving much information about the quality of the generated samples nor the equilibrium properties of the trained models. In our results below, we will show that without putting on the table this information, the comparison between methods or tuning of parameters, becomes extremely unstable.…”

Section: Related Workmentioning

confidence: 99%

Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines

Decelle¹,

Furtlehner²,

Seoane³

2021

Preprint

View full text Add to dashboard Cite

Training Restricted Boltzmann Machines (RBMs) has been challenging for a long time due to the difficulty of computing precisely the log-likelihood gradient. Over the past decades, many works have proposed more or less successful training recipes but without studying the crucial quantity of the problem: the mixing time, i.e. the number of Monte Carlo iterations needed to sample new configurations from a model. In this work, we show that this mixing time plays a crucial role in the dynamics and stability of the trained model, and that RBMs operate in two well-defined regimes, namely equilibrium and out-of-equilibrium, depending on the interplay between this mixing time of the model and the number of steps, k, used to approximate the gradient. We further show empirically that this mixing time increases with the learning, which often implies a transition from one regime to another as soon as k becomes smaller than this time. In particular, we show that using the popular k (persistent) contrastive divergence approaches, with k small, the dynamics of the learned model are extremely slow and often dominated by strong out-of-equilibrium effects. On the contrary, RBMs trained in equilibrium display faster dynamics, and a smooth convergence to dataset-like configurations during the sampling. Finally we discuss how to exploit in practice both regimes depending on the task one aims to fulfill: (i) short k can be used to generate convincing samples in short learning times, (ii) large k (or increasingly large) is needed to learn the correct equilibrium distribution of the RBM.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines

Decelle¹,

Furtlehner²,

Seoane³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A number of recent works have explored the parity dataset using restricted Boltzmann machines (RBMs) and found it to be difficult to learn, even in experiments that train using the entire dataset [ 11 , 21 ]. Recall that an RBM is a universal approximator of distributions on

, given sufficiently many hidden units.…”

Section: Discussionmentioning

confidence: 99%

“…The

dataset can be frustrating to learn for other models, such as restricted Boltzman machines (RBMs) trained with gradient-based methods. The difficulty of training RBMs to learn parity with contrastive divergence and related training algorithms is noted in [ 11 ]. The difficulty for other gradient based deep-learning methods on parity problems has been studied in [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Modeling with Matrix Product States

Stokes

Terilla²

2019

Entropy

View full text Add to dashboard Cite

Inspired by the possibility that generative models based on quantum circuits can provide a useful inductive bias for sequence modeling tasks, we propose an efficient training algorithm for a subset of classically simulable quantum circuit models. The gradient-free algorithm, presented as a sequence of exactly solvable effective models, is a modification of the density matrix renormalization group procedure adapted for learning a probability distribution. The conclusion that circuit-based models offer a useful inductive bias for classical datasets is supported by experimental results on the parity learning problem.

show abstract

“…But unlike CD, PCD keeps a persistent chain to estimate negative gradient. Many CD variants have been proposed to improve the negative gradient estimation as in [13], but almost all based on persistent chain [14], [15], [16].…”

Section: Persistent Contrastive Divergencementioning

confidence: 99%

“…13) where b a and b s are biases of a t and s t+1 respectively; W •F1 and W •F2 are the factorization weights w.r.t. first and second factor respectively; the dynamic bias is defined bŷ b h k = b h k + s t B •k ; and • corresponds to element-wise matrix multiplication.…”

mentioning

confidence: 99%

A Structure of Restricted Boltzmann Machine for Modeling System Dynamics

Padiolleau

Bach

Hugget

et al. 2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

This paper presents a new approach for learning transition function in state representation learning (SRL) for control. While state-of-the-art methods use different deterministic neural networks to learn forward and inverse state transition functions independently with auto-supervised learning, we introduce a bidirectional stochastic model to learn both transition functions. We aim at using the uncertainty of the model on its predictions as an intrinsic motivation for exploration to enhance the representation learning. More, using the same model to learn both transition functions allows sharing the parameters, which can reduce their number and should increase the embedding quality of the representation. We use a factored restricted Boltzmann machine (fRBM) based model, enhanced with dedicated structure for learning system dynamics and transitions with shared parameters. The presented work focuses on building the structure of the bidirectional transition model for unsupervised learning. Our fRBM structure is directly inspired from physics interactions between inputs and outputs in reinforcement learning framework. We compare different training algorithms for learning the model that must be able to predict observable random variables to be used in SRL framework. Our structure is not restricted to any type of observable, nevertheless in this paper we focus on learning dynamics from the OpenAI Gym environment Swinging Pendulum. We show that the proposed structure is able to learn bidirectional transition function and performs well in prediction task.

show abstract

Weighted contrastive divergence

Cited by 15 publications

References 21 publications

Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines

Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines

Probabilistic Modeling with Matrix Product States

A Structure of Restricted Boltzmann Machine for Modeling System Dynamics

Contact Info

Product

Resources

About