Transfer learning using computational intelligence: A survey

Lu, Jie; Behbood, Vahid; Peng, Hao; Zuo, Hua; Xue, Shan; Zhang, Guangquan

doi:10.1016/j.knosys.2015.01.010

Cited by 820 publications

(414 citation statements)

References 65 publications

Supporting

Mentioning

371

Contrasting

Unclassified

Order By: Relevance

“…Because deep neural networks are easily generalizable from one domain to another, the internal representation of the neural network contains no discrimination information on the raw input [37]. Recently, certain transfer learning methods based on deep neural networks have been proposed.…”

Section: Cross-domain Transfer Learningmentioning

confidence: 99%

A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification

et al. 2017

View full text Add to dashboard Cite

Learners in a massive open online course often express feelings, exchange ideas and seek help by posting questions in discussion forums. Due to the very high learner-to-instructor ratios, it is unrealistic to expect instructors to adequately track the forums, find all of the issues that need resolution and understand their urgency and sentiment. In this paper, considering the biases among different courses, we propose a transfer learning framework based on a convolutional neural network and a long short-term memory model, called ConvL, to automatically identify whether a post expresses confusion, determine the urgency and classify the polarity of the sentiment. First, we learn the feature representation for each word by considering the local contextual feature via the convolution operation. Second, we learn the post representation from the features extracted through the convolution operation via the LSTM model, which considers the long-term temporal semantic relationships of features. Third, we investigate the possibility of transferring parameters from a model trained on one course to another course and the subsequent fine-tuning. Experiments on three real-world MOOC courses confirm the effectiveness of our framework. This work suggests that our model can potentially significantly increase the effectiveness of monitoring MOOC forums in real time.

show abstract

Section: Cross-domain Transfer Learningmentioning

confidence: 99%

A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Transfer learning and domain adaptation [3,4] are challenging research areas in recent years and they have been comprehensively studied from various perspectives, including natural language processing [10,14], statistics and machine learning [12,15], and recently computer vision [16][17][18][19]. Pan et al [4] presented a complete survey of cross-domain learning methods, and discussed the different applications of transfer learning.…”

Section: Related Workmentioning

confidence: 99%

“…There are some examples [1][2][3] in the field of artificial intelligence that do not conform to the general assumption of standard machine learning. This leads to an issue known as the domain shift problem [4], where the training and test sets come from different distributions.…”

Section: Introductionmentioning

confidence: 99%

Exploiting kernel-based feature weighting and instance clustering to transfer knowledge across domains

Tahmoresnezhad¹,

Hashemi²

2017

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Learning invariant features across domains is of vital importance to unsupervised domain adaptation, whereclassifiers trained on the training examples (source domain) need to adapt to a different set of test examples (target domain) in which no labeled examples are available. In this paper, we propose a novel approach to find the invariant features in the original space and transfer the knowledge across domains. We extract invariant features of input data by a kernel-based feature weighting approach, which exploits distribution difference and instance clustering to find desired features. The proposed method is called the kernel-based feature weighting (KFW) approach and benefits from the maximum mean discrepancy to measure the difference between domains. KFW uses condensed clusters in the reduced domains, the domains that do not contain variant features, to enhance the classification performance. Simultaneous use of feature weighting and instance clustering increases the adaptation and classification performance. Our approach automatically discovers the invariant features across domains and employs them to bridge between source and target domains. We demonstrate the effectiveness of our approach in the task of artificial and real world dataset examinations.Empirical results show that the proposed method outperforms other state-of-the-art methods on the standard transfer learning benchmark datasets.

show abstract

“…We achieve this through modeling the place and the manner of articulation based on transfer learning. The idea of transfer learning, which should trace back to 20 years ago, has been successfully employed in broad research fields [29]- [32]. This study presents how to employ transfer learning on the articulation modeling of non-native speech with deep neural networks (DNNs).…”

Section: Introductionmentioning

confidence: 99%

Articulatory Modeling for Pronunciation Error Detection without Non-Native Training Data Based on DNN Transfer Learning

Duan

Kawahara

Dantsuji

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYAiming at detecting pronunciation errors produced by second language learners and providing corrective feedbacks related with articulation, we address effective articulatory models based on deep neural network (DNN). Articulatory attributes are defined for manner and place of articulation. In order to efficiently train these models of non-native speech without such data, which is difficult to collect in a large scale, several transfer learning based modeling methods are explored. We first investigate three closely-related secondary tasks which aim at effective learning of DNN articulatory models. We also propose to exploit large speech corpora of native and target language to model inter-language phenomena. This kind of transfer learning can provide a better feature representation of nonnative speech. Related task transfer and language transfer learning are further combined on the network level. Compared with the conventional DNN which is used as the baseline, all proposed methods improved the performance. In the native attribute recognition task, the network-level combination method reduced the recognition error rate by more than 10% relative for all articulatory attributes. The method was also applied to pronunciation error detection in Mandarin Chinese pronunciation learning by Japanese native speakers, and achieved the relative improvement up to 17.0% for detection accuracy and up to 19.9% for F-score, which is also better than the lattice-based combination.

show abstract

Transfer learning using computational intelligence: A survey

Cited by 820 publications

References 65 publications

A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification

A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification

Exploiting kernel-based feature weighting and instance clustering to transfer knowledge across domains

Articulatory Modeling for Pronunciation Error Detection without Non-Native Training Data Based on DNN Transfer Learning

Contact Info

Product

Resources

About