Training restricted Boltzmann machines: An introduction

Fischer, Asja; Igel, Christian

doi:10.1016/j.patcog.2013.05.025

Cited by 405 publications

(213 citation statements)

References 20 publications

Supporting

Mentioning

205

Contrasting

Unclassified

Order By: Relevance

“…Based on the feature that the DBN only learns using the data without labels, two types of data sets are present in this flow chart, namely, training samples without labels and labeled samples for fine tuning [31]. The training samples are used to train the DBN and the SOM; the fine tuning samples are a subset of training samples that have been manually labeled [26,27,32].…”

Section: Network Trainingmentioning

confidence: 99%

The Deep Belief and Self-Organizing Neural Network as a Semi-Supervised Classification Method for Hyperspectral Data

Lan

et al. 2017

Applied Sciences

View full text Add to dashboard Cite

Hyperspectral data is not linearly separable, and it has a high characteristic dimension. This paper proposes a new algorithm that combines a deep belief network based on the Boltzmann machine with a self-organizing neural network. The primary features of the hyperspectral image are extracted with a deep belief network. The weights of the network are fine-tuned using the labeled sample. Feature vectors extracted by the deep belief network are classified by a self-organizing neural network. The method reduces the spectral dimension of the data while preserving the large amount of original information in the data. The method overcomes the long training time required when using self-organizing neural networks for clustering, as well as the training difficulties of Deep Belief Networks (DBN) when the labeled sample size is small, thereby improving the accuracy and robustness of the semi-supervised classification. Simulation results show that the structure of the network can achieve higher classification accuracy when the labeled sample is deficient.

show abstract

Section: Network Trainingmentioning

confidence: 99%

The Deep Belief and Self-Organizing Neural Network as a Semi-Supervised Classification Method for Hyperspectral Data

Lan

et al. 2017

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…We refer the interested reader to Bengio (2009), Hinton (2012, and Fischer & Igel (2014) for detailed derivations. Figure 1 shows a schematic of the the deep-belief network.…”

Section: Robertmentioning

confidence: 99%

“…Figure 1 shows a schematic of the the deep-belief network. The multi-layer DBN can be constructed from several Restricted Boltzmann Machines (Freund & Haussler 1992;Bishop 2006;Le Roux & Bengio 2008;Bengio 2009Bengio , 2012Lee et al 2011a;Hinton 2012;Montavon et al 2012;Fischer & Igel 2014) with the addition of a logistic regression layer at the top of the network. The RBM is a two-layer neural network able to learn the underlying probability distribution over its set of input values.…”

Section: Robertmentioning

confidence: 99%

Dreaming of Atmospheres

Waldmann

2016

ApJ

View full text Add to dashboard Cite

Here, we introduce the RobERt(Robotic Exoplanet Recognition) algorithm for the classification of exoplanetary emission spectra. Spectral retrieval of exoplanetary atmospheres frequently requires the preselection of molecular/ atomic opacities to be defined by the user. In the era of open-source, automated, and self-sufficient retrieval algorithms, manual input should be avoided. User dependent input could, in worst-case scenarios, lead to incomplete models and biases in the retrieval. The RobERtalgorithm is based on deep-belief neural (DBN) networks trained to accurately recognize molecular signatures for a wide range of planets, atmospheric thermal profiles, and compositions. Reconstructions of the learned features, also referred to as the "dreams" of the network, indicate good convergence and an accurate representation of molecular features in the DBN. Using these deep neural networks, we work toward retrieval algorithms that themselves understand the nature of the observed spectra, are able to learn from current and past data, and make sensible qualitative preselections of atmospheric opacities to be used for the quantitative stage of the retrieval process.

show abstract

“…We update the parameters of the subspaceRBM using contrastive divergence learning procedure [8,10]. For this purpose, we need to calculate the gradient of the log-likelihood function.…”

Section: Learningmentioning

confidence: 99%

Learning Invariant Features Using Subspace Restricted Boltzmann Machine

Tomczak

Gonczarek

2016

Neural Process Lett

View full text Add to dashboard Cite

The subspace restricted Boltzmann machine (subspaceRBM) is a third-order Boltzmann machine where multiplicative interactions are between one visible and two hidden units. There are two kinds of hidden units, namely, gate units and subspace units. The subspace units reflect variations of a pattern in data and the gate unit is responsible for activating the subspace units. Additionally, the gate unit can be seen as a pooling feature. We evaluate the behavior of subspaceRBM through experiments with MNIST digit recognition task and Caltech 101 Silhouettes image corpora, measuring cross-entropy reconstruction error and classification error.

show abstract

Training restricted Boltzmann machines: An introduction

Cited by 405 publications

References 20 publications

The Deep Belief and Self-Organizing Neural Network as a Semi-Supervised Classification Method for Hyperspectral Data

The Deep Belief and Self-Organizing Neural Network as a Semi-Supervised Classification Method for Hyperspectral Data

Dreaming of Atmospheres

Learning Invariant Features Using Subspace Restricted Boltzmann Machine

Contact Info

Product

Resources

About