DeNERT-KG: Named Entity and Relation Extraction Model Using DQN, Knowledge Graph, and BERT

Yang, Sung-Min; Yoo, SoYeop; Jeong, Ok‐Ran

doi:10.3390/app10186429

Cited by 21 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bidirectional encoder representation from transformers (BERT) is a pretrained language model 43 . Among these transformers, the algorithm framework can capture the bidirectional relationship in words and sentences 44 .…”

Section: Methodsmentioning

confidence: 99%

Deep learning-based methods for natural hazard named entity recognition

Sun

Liu

Cui

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Natural hazard named entity recognition is a technique used to recognize natural hazard entities from a large number of texts. The method of natural hazard named entity recognition can facilitate acquisition of natural hazards information and provide reference for natural hazard mitigation. The method of named entity recognition has many challenges, such as fast change, multiple types and various forms of named entities. This can introduce difficulties in research of natural hazard named entity recognition. To address the above problem, this paper constructed a natural disaster annotated corpus for training and evaluation model, and selected and compared several deep learning methods based on word vector features. A deep learning method for natural hazard named entity recognition can automatically mine text features and reduce the dependence on manual rules. This paper compares and analyzes the deep learning models from three aspects: pretraining, feature extraction and decoding. A natural hazard named entity recognition method based on deep learning is proposed, namely XLNet-BiLSTM-CRF model. Finally, the research hotspots of natural hazards papers in the past 10 years were obtained through this model. After training, the precision of the XLNet-BilSTM-CRF model is 92.80%, the recall rate is 91.74%, and the F1-score is 92.27%. The results show that this method, which is superior to other methods, can effectively recognize natural hazard named entities.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep learning-based methods for natural hazard named entity recognition

Sun

Liu

Cui

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…We tested state-of-the-art language models like KG-BERT 60 and KGLM 61 that have better MR metrics compared to the graph-embedding models and are generalizable to unseen entities or relations 62 . For example, we obtained an MR of 191 on the validation set by fine-tuning the KG-BERT architecture with the BioBERT as a pre-trained backbone instead of the BERT, which is a significant improvement over the RotatE MR of 1,139.…”

Section: Discussionmentioning

confidence: 99%

FoodAtlas: Automated Knowledge Extraction of Food and Chemicals from Literature

Youn,

Li,

Simmons

et al. 2024

Preprint

View full text Add to dashboard Cite

Automated generation of knowledge graphs that accurately capture published information can help with knowledge organization and access, which have the potential to accelerate discovery and innovation. Here, we present an integrated pipeline to construct a large-scale knowledge graph using large language models in an active learning setting. We apply our pipeline to the association of raw food, ingredients, and chemicals, a domain that lacks such knowledge resources. By using an iterative active learning approach of 4,120 manually curated premise-hypothesis pairs as training data for ten consecutive cycles, the entailment model extracted 230,848 food-chemical composition relationships from 155,260 scientific papers, with 106,082 (46.0%) of them never been reported in any published database. To augment the knowledge incorporated in the knowledge graph, we further incorporated information from 5 external databases and ontology sources. We then applied a link prediction model to identify putative food-chemical relationships that were not part of the constructed knowledge graph. Validation of the 443 hypotheses generated by the link prediction model resulted in 355 new food-chemical relationships, while results show that the model score correlates well (R2 = 0.70) with the probability of a novel finding. This work demonstrates how automated learning from literature at scale can accelerate discovery and support practical applications through reproducible, evidence-based capture of latent interactions of diverse entities, such as food and chemicals.

show abstract

“…Therefore, many deep learning-based biomedical named entity recognition techniques have emerged 6 , mainly including some mainstream models, such as BiLSTM-CRF model 7 , and BiLSTM-CNN-CRF model 8 , etc. ; and advanced models improved on the basis of mainstream models, such as TaughtNet model 9 , DeNERT-KG model 10 , and models that incorporate machine reading comprehension 11,12 . With the development of pre-trained language models, large-scale language models pre-trained on biomedical corpora and fine-tuned on the BioNER dataset 13,14 have great potential to improve biomedical named entity recognition, and can be improved as the availability of training data increases.…”

Section: Introductionmentioning

confidence: 99%

Biomedical named entity recognition based on reinforcement learning label correction

Hao,

Tian,

Tang

et al. 2024

Fourth International Conference on Computer Vision and Data Mining (ICCVDM 2023)

View full text Add to dashboard Cite

Biomedical named entity recognition is of great significance in the field of natural language processing.Deep learning approaches are mainly used at present, and the BERT-BiLSTM-CRF model is one of them. Although the BiLSTM structure in this model can capture bi-directional long-distance dependencies in sentences, there is the problem that the model performance can easily reach the bottleneck by simply adjusting the hyperparameters or changing the network structure, and the quality of its recognition results can hardly be further improved. To address the above problems, this paper adds the label correction process based on deep reinforcement learning to the BERT-BiLSTM-CRF model to form the BERT-BiLSTM-CRF-DRL model. The intelligent body Agent based on deep reinforcement learning is used as a label corrector, and the label correction threshold is set, so that the uncertainty in the annotation results of the BERT-BiLSTMCRF model is greater than the label correction threshold. The labels with uncertainty greater than this threshold in the results are processed for correction. The experimental results show that the F1 values on the datasets BC4CHEMD, BC5CR-CHEM, and NCBI-DISEASE reach 91.07%, 94.02%, and 89.56%,, which are all improved compared with the existing advanced biomedical named entity recognition models.

show abstract

DeNERT-KG: Named Entity and Relation Extraction Model Using DQN, Knowledge Graph, and BERT

Cited by 21 publications

References 23 publications

Deep learning-based methods for natural hazard named entity recognition

Deep learning-based methods for natural hazard named entity recognition

FoodAtlas: Automated Knowledge Extraction of Food and Chemicals from Literature

Biomedical named entity recognition based on reinforcement learning label correction

Contact Info

Product

Resources

About