Research on Task Discovery for Transfer Learning in Deep Neural Networks

Akdemir, Arda

doi:10.18653/v1/2020.acl-srw.6

Cited by 3 publications

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transfer learning is a method using machine learning / deep learning that is commonly used to improve the performance on low-resource tasks. Transfer learning describes the learning schemes when information in source task is used to achieve some improvement in target task performance [23]. Some previous researches on question answering system used this method.…”

Section: Transfer Learningmentioning

confidence: 99%

Transfer Learning for Closed Domain Question Answering in COVID-19

Rachmawati¹,

Yulianti²

2022

IJACSA

View full text Add to dashboard Cite

COVID-19 has been a popular issue around 2019 until today. Recently, there has been a lot of research being conducted to utilize a big amount of data discussing about COVID-19. In this work, we conduct a closed domain question answering (CDQA) task in COVID-19 using transfer learning technique. The transfer learning technique is adopted because a large benchmark for question answering about COVID-19 is still unavailable. Therefore, rich knowledge learned from a large benchmark of open domain QA are utilized using transfer learning to improve the performance of our CDQA system. We use retriever-reader framework for our CDQA system, and propose to use Sequential Dependence Model (SDM) as our retriever component to enhance the effectiveness of the system.Our result shows that the use of SDM retriever can improve the F-1 score of the state-of-the-art baseline CDQA system using BM25 and TF-IDF+cosine similarity retriever by 3,26% and 32,62%, respectively. The optimal parameter settings for our CDQA system are found to be as follows: using 20 top-ranked documents as the retriever's output, five sentences as the passage length, and BERT-Large-Uncased model as the reader. In this optimal parameter setting, SDM retriever can improve the F-1 score of the state-of-the-art baseline CDQA system using BM25 by 5,06 % and TF-IDF+cosine similarity retriever by 24,94 %. Our last experiment then confirms the merit of using transfer learning, since our best-performing model (double fine-tune SQuAD and COVID-QA) is shown to gain eight times higher accuracy than the baseline method without using transfer learning. Further fine-tuning the transfer learning model using closed domain dataset (COVID-QA) can increase the accuracy of the transfer learning model that only fine-tuning with SQuAD by 27, 26%.

show abstract

Section: Transfer Learningmentioning

confidence: 99%

Transfer Learning for Closed Domain Question Answering in COVID-19

Rachmawati¹,

Yulianti²

2022

IJACSA

View full text Add to dashboard Cite

show abstract

Cross-language Transfering the Patent Quality Evaluation Model Based on Active Learning Data Extension

Liu

You

Wang

et al. 2021

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

A new framework for predicting tensile stress of natural rubber based on data augmentation and molecular dynamics simulation data

Hu,

Liu,

Chen

et al. 2024

J Mater Inf

View full text Add to dashboard Cite

This study addresses the challenge of predicting the tensile stress of natural rubber with limited molecular dynamics simulation data, which is a crucial mechanical property for this material. Molecular dynamics (MD) simulations are limited by their scale and computational cost, making it difficult to obtain sufficient data to train machine learning algorithms. To overcome this limitation, we propose a machine learning framework involving three stages: (1) utilizing a Variational Autoencoder (VAE) to rapidly expand the data diversity; (2) employing Ordinary Kriging (OK) to label the VAE-generated virtual samples; and (3) training gradient enhanced regression [Gradient Boosting Regression (GBR)] models by using relevant data on tensile stress in natural rubber. The results demonstrate that the generated data exhibits enhanced rationality, significantly improving the accuracy and reliability of various regression models. This approach provides an effective solution to the problem of data scarcity in MD simulations.

show abstract

Research on Task Discovery for Transfer Learning in Deep Neural Networks

Cited by 3 publications

References 33 publications

Transfer Learning for Closed Domain Question Answering in COVID-19

Transfer Learning for Closed Domain Question Answering in COVID-19

Cross-language Transfering the Patent Quality Evaluation Model Based on Active Learning Data Extension

A new framework for predicting tensile stress of natural rubber based on data augmentation and molecular dynamics simulation data

Contact Info

Product

Resources

About