Fine-tuning BERT for Joint Entity and Relation Extraction in Chinese Medical Text

Kui, Xue; Zhou, Yangming; Ma, Zhiyuan; Ruan, Tong; Zhang, Huan‐Huan; He, Ping

doi:10.1109/bibm47256.2019.8983370

Cited by 70 publications

(46 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Machine learning can, therefore, be applied to the named entity recognition to help automatic tagging [27]. Recent works in using BERT in the NER task [28][29][30] which consider BERT-NER in an open domain are HAREM I Portugese language, and Chinese Medical Text respectively. In our work, we focus on a tourism data set.…”

Section: Deep Learning and Bertmentioning

confidence: 99%

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Chantrapornchai

Tunsakul

2021

ECTI-CIT

View full text Add to dashboard Cite

In this paper, we present two methodologies to extract particular information based on the full text returned from the search engine to facilitate the users. The approaches are based three tasks: name entity recognition (NER), text classiﬁcation and text summarization. The ﬁrst step is the building training data and data cleansing. We consider tourism domain such as restaurant, hotels, shopping and tourism data set crawling from the websites. First, the tourism data are gathered and the vocabularies are built. Several minor steps include sentence extraction, relation and name entity extraction for tagging purpose. These steps are needed for creating proper training data. Then, the recognition model of a given entity type can be built. From the experiments, given review texts, we demonstrate to build the model to extract the desired entity,i.e, name, location, facility as well as relation type, classify the reviews or summarize the reviews. Two tools, SpaCy and BERT, are used to compare the performance of these tasks.

show abstract

Section: Deep Learning and Bertmentioning

confidence: 99%

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Chantrapornchai

Tunsakul

2021

ECTI-CIT

View full text Add to dashboard Cite

show abstract

“…They also compared feature-based and fine-tuning based strategies. Xue et al in [49] fine-tuned the BERT model to focus on the NER and Relation Extraction task words in medical texts. For both the task, a shared parameter layer was employed.…”

Section: Contextualized Embeddings Based Modelsmentioning

confidence: 99%

“…On the other hand, many recent state-of-the-art architectures used CRF layer after a contextual Language Model [31], [45], [49], [51]. Both Li [31] and Yuan et al [51] used BERT-CRF model for sequence labelling tasks.…”

Section: F Conditional Random Fieldmentioning

confidence: 99%

“…In [45], Souza et al employed the BERT model with CRF layer using feature-based and fine-tuning based strategies for Portuguese NER task. Different from Li et al, Yuan et al and Souza et al, Xue et al in [49] fine-tuned the BERT model to focus on the NER and Relation Extraction task specific words in medical texts. For both the tasks, a shared parameter layer was employed.…”

Section: F Conditional Random Fieldmentioning

confidence: 99%

See 1 more Smart Citation

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Ashrafi¹,

Mohammad²,

Mauree³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Named Entity Recognition (NER) is a task in Natural Language Processing (NLP) that aims to classify words into a predetermined list of Named Entities (NE). Many architectures have produced good results on high resourced languages like English and Chinese. However, the NER task has not yet achieved much progress for Bangla, a low resource Language. In this paper, we perform the NER task on Bangla Language using Word2Vec and contextual Bidirectional Encoder Representations from Transformers (BERT) embeddings. We propose multiple BERT-based deep learning models that use the contextualized embedding from BERT as inputs and a simple statistical approach for class weight cost sensitive learning. The modified cost-sensitive loss function was used to address the class imbalance of the data. In our modified cost-sensitive loss function, we penalize the dominant classes by taking the ratio concerning the maximum sample in a class instead of the whole dataset. This penalty is made so that the learner learns slowly for the dominant class. In addition, we experiment by adding a Conditional Random Field (CRF) layer and incorporating Focal Loss to the training process. We found the best F1 Macro score to be 65.96%, F1 Micro score of 90.64%, and F1 Message Understanding Coreference (MUC) score of 72.04%, which were calculated at Named Entity level. Our experimental results demonstrate that one of the proposed models, which jointly optimizes for the CRF loss and class weighted cost-sensitive loss according to our proposed statistical approach, achieve an improvement of over 8% F1 MUC score on a recently introduced Bangla NER dataset when compared to previously published work.

show abstract

“…The joint model has been extensively researched in various NLP tasks, for example, joint entity and relation extraction [ 37 ], joint event extraction and visualization [ 38 ], joint event detection and summarization [ 5 , 39 ], joint event detection and prediction [ 23 ], and joint parsing and name entity recognition [ 40 ]. The key to joint models is designing shared features to capture mutual information between the integrated tasks.…”

Section: Related Workmentioning

confidence: 99%

Continuous Similarity Learning with Shared Neural Semantic Representation for Joint Event Detection and Evolution

Zhou

Luo

Ning

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In the era of the rapid development of today’s Internet, people often feel overwhelmed by vast official news streams or unofficial self-media tweets. To help people obtain the news topics they care about, there is a growing need for systems that can extract important events from this amount of data and construct the evolution procedure of events logically into a story. Most existing methods treat event detection and evolution as two independent subtasks under an integrated pipeline setting. However, the interdependence between these two subtasks is often ignored, which leads to a biased propagation. Besides, due to the limitations of news documents’ semantic representation, the performance of event detection and evolution is still limited. To tackle these problems, we propose a Joint Event Detection and Evolution (JEDE) model, to detect events and discover the event evolution relationships from news streams in this paper. Specifically, the proposed JEDE model is built upon the Siamese network, which first introduces the bidirectional GRU attention network to learn the vector-based semantic representation for news documents shared across two subtask networks. Then, two continuous similarity metrics are learned using stacked neural networks to judge whether two news documents are related to the same event or two events are related to the same story. Furthermore, due to the limited available dataset with ground truths, we make efforts to construct a new dataset, named EDENS, which contains valid labels of events and stories. The experimental results on this newly created dataset demonstrate that, thanks to the shared representation and joint training, the proposed model consistently achieves significant improvements over the baseline methods.

show abstract

Fine-tuning BERT for Joint Entity and Relation Extraction in Chinese Medical Text

Cited by 70 publications

References 18 publications

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Continuous Similarity Learning with Shared Neural Semantic Representation for Joint Event Detection and Evolution

Contact Info

Product

Resources

About