LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition

Liu, Mingyi; Tu, Zhiying; Zhang, Tong; Su, Tonghua; Xu, Xiaofei; Wang, Zhongjie

doi:10.1007/s11063-021-10737-x

Cited by 23 publications

(21 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We follow the AL settings in previous work to achieve consistent evaluation (Kim, 2020;Shelmanov et al, 2021;Liu et al, 2022). Specifically, the unlabeled pool is created by discarding labels from the original training data of each dataset; 2% of which (∼ 242 sentences) is selected for labeling at each iteration for a total of 25 iterations (examples of the first iteration are randomly sampled to serve as the seed D 0 ).…”

Section: Discussionmentioning

confidence: 99%

“…Despite the potential of AL in reducing annotation cost for a target task, most previous AL work focuses on developing data selection strategies to maximize the model performance (Wang and Shang, 2014;Sener and Savarese, 2017;Ash et al, 2019;Kim, 2020;Liu et al, 2022;Margatina et al, 2021). As such, previous AL methods and frameworks tend to ignore the necessary time to train models and perform data selection at each AL iteration that can be significantly long and hinder annotators' productivity and model performance.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

Nguyen¹,

Ngo²,

Min³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents FAMIE, a comprehensive and efficient active learning (AL) toolkit for multilingual information extraction. FAMIE is designed to address a fundamental problem in existing AL frameworks where annotators need to wait for a long time between annotation batches due to the time-consuming nature of model training and data selection at each AL iteration. This hinders the engagement, productivity, and efficiency of annotators. Based on the idea of using a small proxy network for fast data selection, we introduce a novel knowledge distillation mechanism to synchronize the proxy network with the main large model (i.e., BERT-based) to ensure the appropriateness of the selected annotation examples for the main model. Our AL framework can support multiple languages.The experiments demonstrate the advantages of FAMIE in terms of competitive performance and time efficiency for sequence labeling with AL. We publicly release our code (https://github.com/ nlp-uoregon/famie) and demo website (http://nlp.uoregon.edu:9000/). A demo video for FAMIE is provided at: https://youtu.be/I2i8n_jAyrY.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

Nguyen¹,

Ngo²,

Min³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…According to results from previous research, sequence level measures are superior to aggregating token-level information for sequence-labeling with CRF models (Settles and Craven, 2008;Chen et al, 2015b;Shen et al, 2017;Liu et al, 2020). We incorporate the following most representative query methods that are explored in prior work for NER tasks (Settles and Craven, 2008;Chen et al, 2015b;Shen et al, 2017;Chen et al, 2017;Siddhant and Lipton, 2018;Shelmanov et al, 2019;Chaudhary et al, 2019;Grießhaber et al, 2020;Shui et al, 2020;Ren et al, 2021;Liu et al, 2020Liu et al, , 2022Agrawal et al, 2021), in our experiments:…”

Section: Active Learning Withmentioning

confidence: 99%

“…In transfer learning, models transfer knowledge learned from data-rich languages or tasks to languages or tasks with less or no annotated data (Wang et al, 2019;Lauscher et al, 2020;Xie et al, 2018a;Yuan et al, 2019;Pires et al, 2019;Xie et al, 2018b;Plank, 2019). Active learning is an approach to maximize the utility of annotations while minimizing the annotation effort on the unlabeled target data (Chen et al, 2015a;Miller et al, 2019;Liu et al, 2020Liu et al, , 2022Chaudhary et al, 2019;Shelmanov et al, 2019;Lauscher et al, 2020). We train a German biomedical NER model building on these two approaches, addressing the following research questions: a) How to transfer knowledge from annotated English clinical narratives corpora to the German NER model?…”

Section: Introductionmentioning

confidence: 99%

Cross-lingual German Biomedical Information Extraction: from Zero-shot to Human-in-the-Loop

Liang¹,

Hartmann²,

Sonntag³

2023

Preprint

View full text Add to dashboard Cite

This paper presents our project proposal for extracting biomedical information from German clinical narratives with limited amounts of annotations. We first describe the applied strategies in transfer learning and active learning for solving our problem. After that, we discuss the design of the user interface for both supplying model inspection and obtaining user annotations in the interactive environment.

show abstract

“…The results showed, with the aid of AL and merely one-fourth of the training dataset, the model achieved 99% accuracy of the best deep learning models trained on the whole dataset. In (Liu, Tu, Wang, & Xu, 2020), using the BERT-CRF model, an uncertainty-based AL strategy was applied to NER and achieved satisfactory results.…”

Section: Active Learningmentioning

confidence: 99%

Improving Question Answering Performance Using Knowledge Distillation and Active Learning

Boreshban¹,

Mirbostani²,

Ghassem-Sani³

et al. 2021

Preprint

View full text Add to dashboard Cite

Contemporary question answering (QA) systems, including transformer-based architectures, suffer from increasing computational and model complexity which render them inefficient for real-world applications with limited resources. Further, training or even finetuning such models requires a vast amount of labeled data which is often not available for the task at hand. In this manuscript, we conduct a comprehensive analysis of the mentioned challenges and introduce suitable countermeasures. We propose a novel knowledge distillation (KD) approach to reduce the parameter and model complexity of a pre-trained BERT system and utilize multiple active learning (AL) strategies for immense reduction in annotation efforts. In particular, we demonstrate that our model achieves the performance of a 6-layer TinyBERT and DistilBERT, whilst using only 2% of their total parameters. Finally, by the integration of our AL approaches into the BERT framework, we show that state-of-the-art results on the SQuAD dataset can be achieved when we only use 20% of the training data.

show abstract

LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition

Cited by 23 publications

References 29 publications

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

Cross-lingual German Biomedical Information Extraction: from Zero-shot to Human-in-the-Loop

Improving Question Answering Performance Using Knowledge Distillation and Active Learning

Contact Info

Product

Resources

About