Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios

Abstract

Domain adaptation is an effective solution to data scarcity in low-resource scenarios. However, when applied to token-level tasks such as bioNER, domain adaptation methods often suffer from the challenging linguistic characteristics that clinical narratives possess, which leads to unsatsifactory performance. In this paper, we present a simple yet effective hardnessguided domain adaptation (HGDA) framework for bioNER tasks that can effectively leverage the domain hardness information to improve the adaptability of the learnt model in the low-resource scenarios. Experimental results on biomedical datasets show that our model can achieve significant performance improvement over the recently published state-of-theart (SOTA) MetaNER model.

Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification

“…After each AL iteration, we fine-tune these models using random re-initialization (Frankle and Carbin 2019) for enhanced efficacy-a method found to be more effective than incremental fine-tuning with new samples (Gal, Islam, and Ghahramani 2017). The fine-tuning process, executed on RTX3090 GPUs, sets a maximum sequence length of 256, runs for up to 80 epochs, and uses the AdamW optimizer with a 1e-5 learning rate (Nguyen et al 2022(Nguyen et al , 2023a. Within the AL setup, the initial training and validation sets are composed of 100 and 1000 samples from the training set.…”