From neural PCA to deep unsupervised learning

Valpola, Harri

doi:10.1016/b978-0-12-802806-3.00008-7

Cited by 135 publications

(86 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2). Our model bears some resemblance to the Ladder Network [9] which is also a hierarchical latent variable model where inference results in an autoencoder with skip connections. Our work differs substantially from that work in how inference is performed.…”

Section: Methodsmentioning

confidence: 99%

PHiSeg: Capturing Uncertainty in Medical Image Segmentation

Baumgartner

Tezcan

Chaitanya

et al. 2019

Lecture Notes in Computer Science

143

118

View full text Add to dashboard Cite

Segmentation of anatomical structures and pathologies is inherently ambiguous. For instance, structure borders may not be clearly visible or different experts may have different styles of annotating. The majority of current state-of-the-art methods do not account for such ambiguities but rather learn a single mapping from image to segmentation. In this work, we propose a novel method to model the conditional probability distribution of the segmentations given an input image. We derive a hierarchical probabilistic model, in which separate latent variables are responsible for modelling the segmentation at different resolutions. Inference in this model can be efficiently performed using the variational autoencoder framework. We show that our proposed method can be used to generate significantly more realistic and diverse segmentation samples compared to recent related work, both, when trained with annotations from a single or multiple annotators. The code for this paper is freely available at https://github.com/baumgach/PHiSeg-code.

show abstract

Section: Methodsmentioning

confidence: 99%

PHiSeg: Capturing Uncertainty in Medical Image Segmentation

Baumgartner

Tezcan

Chaitanya

et al. 2019

Lecture Notes in Computer Science

143

118

View full text Add to dashboard Cite

show abstract

“…The method bears some similarities to the so called Ladder Networks, introduced in [28] and used in the context of semisupervised learning in [29]. However, our method explicitly optimises the combined supervised and unsupervised criteria for each layer of the network independently, whereas, in Ladder Networks the optimisation is performed for all layers conjunctively.…”

Section: Introductionmentioning

confidence: 99%

Sparse Autoencoder Based Semi-Supervised Learning for Phone Classification with Limited Annotations

Dhaka¹,

Salvi²

2017

GLU 2017 International Workshop on Grounding Language Understanding

View full text Add to dashboard Cite

We propose the application of a semi-supervised learning method to improve the performance of acoustic modelling for automatic speech recognition with limited linguistically annotated material. Our method combines sparse autoencoders with feed-forward networks, thus taking advantage of both unlabelled and labelled data simultaneously through mini-batch stochastic gradient descent. We tested the method with varying proportions of labelled vs unlabelled observations in framebased phoneme classification on the TIMIT database. Our experiments show that the method outperforms standard supervised models of similar complexity for an equal amount of labelled data and provides competitive error rates compared to state-of-the-art graph-based semi-supervised learning techniques.

show abstract

“…1 In Kingma et al [11], semi-supervised learning was used with deep generative models, showing state-of-the-art performance on the MNIST data set. In Rasmus et al [12], unsupervised Ladder networks [13] were extended by adding a supervised learning component. Their resulting model reached state-of-the-art performance on both MNIST and CIFAR-10.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised pre-training for fully convolutional neural networks

Wiehman

Kroon

Villiers

2016

2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech)

View full text Add to dashboard Cite

Abstract-Unsupervised pre-training of neural networks has been shown to act as a regularization technique, improving performance and reducing model variance. Recently, fully convolutional networks (FCNs) have shown state-of-the-art results on various semantic segmentation tasks. Unfortunately, there is no efficient approach available for FCNs to benefit from unsupervised pre-training. Given the unique property of FCNs to output segmentation maps, we explore a novel variation of unsupervised pre-training specifically designed for FCNs. We extend an existing FCN, called U-net, to facilitate end-to-end unsupervised pre-training and apply it on the ISBI 2012 EM segmentation challenge data set. We performed a battery of significance tests for both equality of means and equality of variance, and show that our results are consistent with previous work on unsupervised pre-training obtained from much smaller networks. We conclude that end-to-end unsupervised pre-training for FCNs adds robustness to random initialization, thus reducing model variance.

show abstract

From neural PCA to deep unsupervised learning

Cited by 135 publications

References 21 publications

PHiSeg: Capturing Uncertainty in Medical Image Segmentation

PHiSeg: Capturing Uncertainty in Medical Image Segmentation

Sparse Autoencoder Based Semi-Supervised Learning for Phone Classification with Limited Annotations

Unsupervised pre-training for fully convolutional neural networks

Contact Info

Product

Resources

About