GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

Abstract

Instead of using expensive manual annotations, researchers have proposed to train named entity recognition (NER) systems using heuristic labeling rules. However, devising labeling rules is challenging because it often requires a considerable amount of manual effort and domain expertise. To alleviate this problem, we propose GLARA, a graph-based labeling rule augmentation framework, to learn new labeling rules from unlabeled data. We first create a graph with nodes representing candidate rules extracted from unlabeled data. Then, we design a new graph neural network to augment labeling rules by exploring the semantic relations between rules. We finally apply the augmented rules on unlabeled data to generate weak labels and train a NER model using the weakly labeled data. We evaluate our method on three NER datasets and find that we can achieve an average improvement of +20% F1 score over the best baseline when given a small set of seed rules. ⋮ Labeling Rule Applier *noma -> Disease *athy -> Disease *tion -> Other *lity -> Other Seeding Rules *noma-> Disease *athy -> Disease *homa -> Disease *kemias-> Disease *ndrome-> Disease Rank and Select New Rules ... *noma *kemias *ation *tion *lity *ility *athy *ndrome *homa *itoyl *ation *ency *trophy *onia *noma *tonia *kemias *tity *itity *tion *ndrome *homa *ndrome Candidate Rules

Information extraction from electronic medical documents: state of the art and future research directions

“…Zhao et al [161] propose a weakly supervised method where they manually prepare some seeding rules and automatically extract all possible rules from unlabeled text for each of the six rule types, and connect them in a graph using cosine similarity. Note that the rule is represented by the average contextual embedding of its matched candidate entities.…”

“…However, this method is restricted by the entity labeling granularity where we can find some nested entities. The method of Zhao et al [161] avoids the ambiguity as it automatically propagates some seed rules based on lexical or contextual clues which are strong indicators of entity recognition. In addition, the authors have fine-tuned a pre-trained contextual embedding model BERT in the biomedical domain.…”

Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming

“…The methods generally require an initial set of labeled data, or seed LFs developed by users. Snuba [33] learns weak classifiers as heuristic models from a small labeled dataset; TALLOR [16] and GLaRA [39] use an initial set of seed LFs to generate new ones by compounding multiple simpler LFs and by exploiting the semantic relationship of the seed LFs respectively; [31] applies program systhesis to generate task-level LFs from a set of labeled data and domain-level LFs.…”

WRENCH: A Comprehensive Benchmark for Weak Supervision

“…[7] and [25] interactively generate labeling functions based on user feedback. TALLOR [46] and GLaRA [106] automatically augment an initial set of labeling functions with new ones. Different from existing works that optimize the task performance, the procedural labeling function generators in WRENCH facilitate the study of the impact of different weak supervision sources.…”

“…(2) Active generation and repurposing of supervision sources. To further reduce human annotation efforts, very recently, researchers turn to active generation [91,46,106,7,25] and repurposing [27] of supervision sources. In the future, we plan to incorporate these new tasks and methods into WRENCH to extend its scope.…”