Deep Learning via Semi-supervised Embedding

Weston, Jason; Ratle, Frédéric; Mobahi, Hossein; Collobert, Ronan

doi:10.1007/978-3-642-35289-8_34

Cited by 545 publications

(469 citation statements)

References 18 publications

Supporting

Mentioning

452

Contrasting

Unclassified

Order By: Relevance

“…sequence data, with complex dependencies between data nodes. Weston et al [34] propose a TDNN [35] method for NLP tasks, where there are sequence data involved. In the work of [36], the authors apply CNN for object recognition task in video.…”

Section: Related Workmentioning

confidence: 99%

Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

Liu

Zhang

et al. 2010

2010 IEEE International Conference on Data Mining

View full text Add to dashboard Cite

Recently, combining Conditional Random Fields (CRF) with Neural Network has shown the success of learning high-level features in sequence labeling tasks. However, such models are difficult to train because of the increase of the parameters to tune which needs enormous of labeled data to avoid overfitting. In this paper, we propose a transfer learning framework for the sequence labeling task of gesture recognition. Taking advantage of the frame correlation, we design an unsupervised sequence model as a pseudo auxiliary task to capture the underlying information from both the labeled and unlabeled data. The knowledge learnt by the auxiliary task can be transferred to the main task of CRF with a deep architecture by sharing the hidden layers, which is very helpful for learning meaningful representation and reducing the need of labeled data. We evaluate our model under 3 gesture recognition datasets. The experimental results of both supervised learning and semi-supervised learning show that the proposed model improves the performance of the CRF with Neural Network and other baseline models.

show abstract

Section: Related Workmentioning

confidence: 99%

Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

Liu

Zhang

et al. 2010

2010 IEEE International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…Although most recent breakthroughs have been achieved with applications of supervised learning, the potential added value of unsupervised learning is so important that it is worthwhile exploring a large array of approaches. The main appeal of unsupervised learning is mostly that it is a crucial ingredient in semi-supervised learning (Weston, Ratle, & Collobert, 2008): there are many more data sources that are unlabeled than data sources that are labeled, and the volume of the unlabeled ones can be considerably larger. Similarly, better unsupervised representation learning has already been shown its advantage as a regularizer (Bengio, Lamblin, Popovici, & Larochelle, 2007;Erhan et al, 2010;Hinton, Osindero, & Teh, 2006;Le et al, 2012;Lee, Ekanadham, & Ng, 2008;Lee, Grosse, Ranganath, & Ng, 2009a;Raina, Battle, Lee, Packer, & Ng, 2007;Ranzato, Poultney, Chopra, & LeCun, 2007) and in the context of transfer learning, e.g., winning two transfer learning competitions in 2011 Mesnil et al, 2011), and domain adaptation (Glorot, Bordes, & Bengio, 2011b).…”

Section: Unsupervised Learningmentioning

confidence: 99%

Editorial introduction to the Neural Networks special issue on Deep Learning of Representations

Bengio¹,

Lee²

2015

Neural Networks

View full text Add to dashboard Cite

“…The learning of p(X) can be performed either independently [20,21] or jointly [22,23] to the learning of p(y|X). These approaches have shown to be efficient on numerous problems such as natural language processing [24], speech recognition [25] or handwriting recognition [26].…”

Section: Global Image Labeling Approachesmentioning

confidence: 99%

IODA: An input/output deep architecture for image labeling

Lerouge

Hérault

Chatelain

et al. 2015

Pattern Recognition

View full text Add to dashboard Cite

In this article, we propose a deep neural network (DNN) architecture called Input Output Deep Architecture (IODA) for solving the problem of image labeling. IODA directly links a whole image to a whole label map, assigning a label to each pixel using a unique neural network forward. Instead of designing a handcrafted a priori model on labels (such as an atlas in the medical domain), we propose to automatically learn the dependencies between labels. The originality of IODA is to transpose DNN pre-training input trick to outputs, in order to learn a high level representation of labels. It allows a fast image labeling inside a fully neural network framework, without the need of any preprocessing such as feature designing or output coding. In this article, IODA is applied on both a toy texture problem and a real-world medical image dataset, showing promising results. We provide an open source implementation of IODA. AbstractIn this article, we propose a deep neural network (DNN) architecture called Input Output Deep Architecture (IODA) for solving the problem of image labeling. IODA directly links a whole image to a whole label map, assigning a label to each pixel using a single neural network forward step. Instead of designing a handcrafted a priori model on labels (such as an atlas in the medical domain), we propose to automatically learn the dependencies between labels. The originality of IODA is to transpose DNN input pre-training trick to the output space, in order to learn a high level representation of labels. It allows a fast image labeling inside a fully neural network framework, without the need of any preprocessing such as feature designing or output coding.In this article, IODA is applied on both a toy texture problem and a realworld medical image dataset, showing promising results. We provide an open source implementation of IODA 12 .

show abstract

Deep Learning via Semi-supervised Embedding

Cited by 545 publications

References 18 publications

Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

Editorial introduction to the Neural Networks special issue on Deep Learning of Representations

IODA: An input/output deep architecture for image labeling

Contact Info

Product

Resources

About