Composite Denoising Autoencoders

Geras, Krzysztof J.; Sutton, Charles

doi:10.1007/978-3-319-46128-1_43

Cited by 10 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although DA provides a representation which is supposedly more robust to noise, the learnt parameters are susceptible to the level of noise applied to the original vectors; high‐level noise will harm the learning of robust representations . To overcome this issue, a scheduled DA (ScheDA) has been introduced, in which the network is trained on gradually decreasing noise level for corrupting feature vectors . Initially, a high level of noise forces the network to learn global and coarse‐grained features.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Then, by decreasing the noise levels, finer representations are learnt. Alternatively, c omposite DAs are proposed, in which, at each stage of the training, data with specific noise level is presented to the network, and only a subset of hidden layers is tuned …”

Section: Literature Reviewmentioning

confidence: 99%

“…Alternatively, composite DAs are proposed, in which, at each stage of the training, data with specific noise level is presented to the network, and only a subset of hidden layers is tuned. 16 DAs have been employed for cross-language sentiment classification tasks where there is a lack of labels on the target language. 17 In this method, a machine translation was used to generate texts from the source language (English) to the target language (Chinese).…”

Section: Denoising Autoencodersmentioning

confidence: 99%

See 2 more Smart Citations

Stacked denoising autoencoders for sentiment analysis: a review

Sagha

Cummins

Schuller

2017

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Deep learning has been shown to outperform numerous conventional machine learning algorithms (e.g., support vector machines) in many fields, such as image processing and text analyses. This is due to its outstanding capability to model complex data distributions. However, as networks become deeper, there is an increased risk of overfitting and higher sensitivity to noise. Stacked denoising autoencoders (SDAs) provide an infrastructure to resolve these issues. In the field of sentiment recognition from textual contents, SDAs have been widely used (especially for domain adaptation), and have been consistently refined and improved through defining new alternate topologies as well as different learning algorithms. A wide selection of these approaches are reviewed and compared in this article. The results coming from the reviewed works indicate the promising capability of SDAs to perform sentiment recognition on a multitude of domains and languages. WIREs Data Mining Knowl Discov 2017, 7:e1212. doi: 10.1002/widm.1212 This article is categorized under: Algorithmic Development > Text Mining Technologies > Machine Learning

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Denoising Autoencodersmentioning

confidence: 99%

See 1 more Smart Citation

Stacked denoising autoencoders for sentiment analysis: a review

Sagha

Cummins

Schuller

2017

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…A DAE is an autoencoder trained to reconstruct the signal that was corrupted with artificial noise. [34][35][36][37] Previously, Alain and Bengio 38 and Nguyen et al 39 used DAE to construct generative models, and pointed out that the output of an optimal DAE is a local mean of the true data density, and the autoencoder error (the difference between its output and input) is a mean shift vector, 40 where the expectation is over all images u and Gaussian noise with standard variance . It should be pointed that the formulation in Equation 2 is resemble to the nonlocal total variation and block-matching-and-3D-filtering (BM3D) prior.…”

Section: Daepmentioning

confidence: 99%

Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors

Liu

Yang

Cheng

et al. 2019

Magnetic Resonance in Med

View full text Add to dashboard Cite

Purpose Although recent deep learning methodologies have shown promising results in fast MR imaging, how to explore it to learn an explicit prior and leverage it into the observation constraint is still desired. Methods A denoising autoencoder (DAE) network is leveraged as an explicit prior to address the highly undersampling MR image reconstruction problem. First, inspired by the observation that the prior information learned from high‐dimension signals is more effective than that from the low‐dimension counterpart in image restoration tasks, we train the network in a multichannel scenario and apply the learned network to single‐channel image reconstruction by a variables augmentation technique. Second, because of the fact that multiple implementations of artificial noise generation in DAE favors a better underlying result, we introduce a 2‐sigma rule to complement each other for improving the final reconstruction. The whole algorithm is tackled by proximal gradient descent. Results Experimental results under varying sampling trajectories and acceleration factors consistently demonstrate the superiority of the enhanced autoencoding priors, in terms of peak signal‐to‐noise ratio, structural similarity, and high‐frequency error norm. Conclusion A simple and effective way to incorporate the DAE prior into highly undersampling MR reconstruction is proposed. Once the DAE prior is obtained, it can be applied to the reconstruction tasks with different sampling trajectories and acceleration factors, and achieves superior performance in comparison with state‐of‐the‐art methods.

show abstract

“…Denoising Autoencoder (DAE) [1][2][3][4][5] is an extension of the classical autoencoder [6,7], where feature denoising is key for the autoencoder to generate better features. In contrast to the classic autoencoder, the input vector in DAE is first corrupted by randomly setting some of features to zero.…”

Section: Introductionmentioning

confidence: 99%

Instance-Wise Denoising Autoencoder for High Dimensional Data

Chen

2016

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Denoising Autoencoder (DAE) is one of the most popular fashions that has reported significant success in recent neural network research. To be specific, DAE randomly corrupts some features of the data to zero as to utilize the cooccurrence information while avoiding overfitting. However, existing DAE approaches do not fare well on sparse and high dimensional data. In this paper, we present a Denoising Autoencoder labeled here as Instance-Wise Denoising Autoencoder (IDA), which is designed to work with high dimensional and sparse data by utilizing the instance-wise cooccurrence relation instead of the feature-wise one. IDA works ahead based on the following corruption rule: if an instance vector of nonzero feature is selected, it is forced to become a zero vector. To avoid serious information loss in the event that too many instances are discarded, an ensemble of multiple independent autoencoders built on different corrupted versions of the data is considered. Extensive experimental results on high dimensional and sparse text data show the superiority of IDA in efficiency and effectiveness. IDA is also experimented on the heterogenous transfer learning setting and cross-modal retrieval to study its generality on heterogeneous feature representation.

show abstract

Composite Denoising Autoencoders

Cited by 10 publications

References 11 publications

Stacked denoising autoencoders for sentiment analysis: a review

Stacked denoising autoencoders for sentiment analysis: a review

Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors

Instance-Wise Denoising Autoencoder for High Dimensional Data

Contact Info

Product

Resources

About