A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks

Semwal, Tushar; Mathur, Gaurav; Yenigalla, Promod; Nair, Shivashankar B.

doi:10.1137/1.9781611975321.58

Cited by 38 publications

(26 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In such applications, computer vision models pre-trained on a very large but general image data (e.g., ImageNet) are exploited to transfer knowledge to a specialized clinical imaging dataset which is relatively small but sufficient for domain-driven fine-tuning of the general trained model. The success of applying transfer learning on image applications, opened up the possibility to exploit transfer learning in non-clinical NLP applications, such as sentiment classification [42]. However, applying transfer learning of DL models to clinical NLP tasks is still an understudied research topic.…”

Section: Transfer Learningmentioning

confidence: 99%

Privacy-Preserving Deep Learning NLP Models for Cancer Registries

Alawad

Yoon

Gao

et al. 2021

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

Population cancer registries can benefit from Deep Learning (DL) to automatically extract cancer characteristics from the high volume of unstructured pathology text reports they process annually. The success of DL to tackle this and other real-world problems is proportional to the availability of large labeled datasets for model training. Although collaboration among cancer registries is essential to fully exploit the promise of DL, privacy and confidentiality concerns are main obstacles for data sharing across cancer registries. Moreover, DL for natural language processing (NLP) requires sharing a vocabulary dictionary for the embedding layer which may contain patient identifiers. Thus, even distributing the trained models across cancer registries causes a privacy violation issue. In this paper, we propose DL NLP model distribution via privacy-preserving transfer learning approaches without sharing sensitive data. These approaches are used to distribute a multitask convolutional neural network (MT-CNN) NLP model among cancer registries. The model is trained to extract six key cancer characteristics-tumor site, subsite, laterality, behavior, histology, and grade-from cancer pathology reports. Using 410,064 pathology documents from two cancer registries, we compare our proposed approach to conventional transfer learning without privacy-preserving, single-registry models, and a model trained on centrally hosted data. The results show that transfer learning approaches including data sharing and model distribution outperform significantly the single-registry model. In addition, the best performing privacy-preserving model distribution approach achieves statistically indistinguishable average micro-and macro-F1 scores across all extraction tasks (0.823,0.580) as compared to the centralized model (0.827,0.585).

show abstract

Section: Transfer Learningmentioning

confidence: 99%

Privacy-Preserving Deep Learning NLP Models for Cancer Registries

Alawad

Yoon

Gao

et al. 2021

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

show abstract

“…This information is used in transfer learning algorithms [45][46][47] to develop knowledge of surrounding rocks and their chemical makeup in upcoming Mars missions (Mars Perseverance Rover, its probes and helicopter) and in future outer and deep space missions (satellites, helicopters and rovers). Further, transfer learning will be used to effectively process source logic networks to target domain and then revises the mapped structure to further improve its accuracy [48].…”

Section: Transfer Learningmentioning

confidence: 99%

Modelling and Simulation of Microstructural Evolution for Extra-terrestrial Planetary Chemistry Mapping Using Artificial Intelligence – a NASA Mars Rover Study

Rafique¹

2020

Preprint

View full text Add to dashboard Cite

Development of rovers and development of infrastructure which enables them to probe other planets (such as Mars) have sparked a lot of interest recently specially with increasing public attention in Moon and Mars program by National Aeronautics and Space Administration. This is designed to be achieved by various means such as advanced spectroscopy and artificial intelligent techniques such as deep learning and transfer learning to enable the rover to not only map the surface of planet but to get a detailed information about its chemical makeup in layers beneath (deep learning) and in areas around point of observation (transfer learning). In this work, which is part of a proposal, later approach is explored. A systematic strategy is presented which make use of aforementioned techniques developed for metallic glass matrix composites as benchmark and helps develop algorithms for chemistry mapping of actual Martian surface on Perseverance Rover launching shortly.

show abstract

“…One way to understand the effect of each of these layers is to finetune or freeze these layers during model transfer and report the best performing model. However, as suggested by [29], the best performance is realized when all layers of a pre-trained model on D S are transferred and the model is led to fine-tune itself using D G . Therefore, we follow the same practice and let the transferred model fine-tune all trainable variables in our model.…”

Section: Transferring Network Layersmentioning

confidence: 99%

Deep Transfer Reinforcement Learning for Text Summarization

Keneshloo¹,

Ramakrishnan²,

Reddy³

2019

Proceedings of the 2019 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Deep neural networks are data hungry models and thus face difficulties when attempting to train on small text datasets. Transfer learning is a potential solution but their effectiveness in the text domain is not as explored as in areas such as image analysis. In this paper, we study the problem of transfer learning for text summarization and discuss why existing state-of-the-art models fail to generalize well on other (unseen) datasets. We propose a reinforcement learning framework based on a self-critic policy gradient approach which achieves good generalization and state-ofthe-art results on a variety of datasets. Through an extensive set of experiments, we also show the ability of our proposed framework to fine-tune the text summarization model using only a few training samples. To the best of our knowledge, this is the first work that studies transfer learning in text summarization and provides a generic solution that works well on unseen data.

show abstract

A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks

Cited by 38 publications

References 10 publications

Privacy-Preserving Deep Learning NLP Models for Cancer Registries

Privacy-Preserving Deep Learning NLP Models for Cancer Registries

Modelling and Simulation of Microstructural Evolution for Extra-terrestrial Planetary Chemistry Mapping Using Artificial Intelligence – a NASA Mars Rover Study

Deep Transfer Reinforcement Learning for Text Summarization

Contact Info

Product

Resources

About