Loss-based Active Learning for Named Entity Recognition

Linh, Le Thai; Nguyen, Minh-Tien; Zuccon, Guido; Demartini, Gianluca

doi:10.1109/ijcnn52387.2021.9533675

Cited by 4 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The outputs of the model may be, for example, the entropy (Dagan and Engelson 1995), the confidence of the prediction (Culotta and McCallum 2005), the margin between the confidence of the two highest predicted classes (Settles 2012), the information benefit from the Bayesian model's parameters (Gal, Islam1, and Ghahramani 2017), the ensemble of multiple variances of uncertainty AL methods for image input (Beluch et al 2018). Recent uncertainty-based algorithms are based on the loss of the target model (Yoo and Kweon 2019;Linh et al 2021). However, these approaches often select outliers due to their high uncertainty (Parvaneh et al 2022).…”

Section: Uncertainty-based Almentioning

confidence: 99%

“…The most common AL algorithm family is that of uncertainty-based algorithms, which estimate the "informativeness" of an instance (Settles 2012). These algorithms select instances that the target model is most uncertain about (Siddhant and Lipton 2018;Yoo and Kweon 2019;Linh et al 2021;Ostapuk, Yang, and Cudre-Mauroux 2019). Another AL family is distributionbased AL.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CoLAL: Co-learning Active Learning for Text Classification

Le,

Zhao,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

In the machine learning field, the challenge of effectively learning with limited data has become increasingly crucial. Active Learning (AL) algorithms play a significant role in this by enhancing model performance. We introduce a novel AL algorithm, termed Co-learning (CoLAL), designed to select the most diverse and representative samples within a training dataset. This approach utilizes noisy labels and predictions made by the primary model on unlabeled data. By leveraging a probabilistic graphical model, we combine two multi-class classifiers into a binary one. This classifier determines if both the main and the peer models agree on a prediction. If they do, the unlabeled sample is assumed to be easy to classify and is thus not beneficial to increase the target model's performance. We prioritize data that represents the unlabeled set without overlapping decision boundaries. The discrepancies between these boundaries can be estimated by the probability that two models result in the same prediction. Through theoretical analysis and experimental validation, we reveal that the integration of noisy labels into the peer model effectively identifies target model's potential inaccuracies. We evaluated the CoLAL method across seven benchmark datasets: four text datasets (AGNews, DBPedia, PubMed, SST-2) and text-based state-of-the-art (SOTA) baselines, and three image datasets (CIFAR100, MNIST, OpenML-155) and computer vision SOTA baselines. The results show that our CoLAL method significantly outperforms existing SOTA in text-based AL, and is competitive with SOTA image-based AL techniques.

show abstract