On Dataset Transferability in Active Learning for Transformers

Abstract

Active learning (AL) aims to reduce labeling costs by querying the examples most beneficial for model learning. While the effectiveness of AL for fine-tuning transformer-based pre-trained language models (PLMs) has been demonstrated, it is less clear to what extent the AL gains obtained with one model transfer to others. We consider the problem of transferability of actively acquired datasets in text classification and investigate whether AL gains persist when a dataset built using AL coupled with a specific PLM is used to train a different PLM. We link the AL dataset transferability to the similarity of instances queried by the different PLMs and show that AL methods with similar acquisition sequences produce highly transferable datasets regardless of the models used. Additionally, we show that the similarity of acquisition sequences is influenced more by the choice of the AL method than the choice of the model.

A Survey on Deep Active Learning: Recent Advances and New Frontiers

“…As shown in Table III, the integration of DL and AL is leading to an increasing application of AL methods in various domains of life, ranging from agricultural development [82] to industrial revitalization [82] and from artificial intelligence [137] to biomedical fields [160]. In this section, we aim to provide a systematic and detailed overview of existing DAL-related work from a broad application perspective.…”

“…Interestingly, they achieve comparable performance in widely used text classification datasets while training in less than 20% of the labeled data, which demonstrates their ability to utilize limited labeled data. In another study, Jelenic et al [137] conduct an initial empirical study to investigate the transferability of the DAL by using PLMs. They find that DAL can effectively adapt to new datasets with pretrained models.…”