“…In recent years, however, RBM-based pre-training has been largely abandoned, because direct supervised training of deep neural networks has improved due to new techniques such as better initialization [3], non-saturating activation functions [4], and better control of generalization [5]. However, very recent work has begun to reconsider the value of unsupervised pre-training, specifically in the context of representation learning on a large set of unlabeled data, for use in supervised training on a smaller set of labeled data [6,7,8].…”