On Robustness and Bias Analysis of BERT-Based Relation Extraction

Li, Luoqiu; Chen, Xiang; Ye, Hongbin; Bi, Zhen; Zhang, Ningyu; Chen, Huajun

doi:10.1007/978-981-16-6471-7_4

Cited by 7 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The utilization of a permutation dataset allows for the simulation of data variations, thereby providing insights into the generalizability of models when confronted with new data [14] , [32] . In this study, we employed the manually created permutation test set to investigate the impact of data variances on model performance.…”

Section: Methodsmentioning

confidence: 99%

“…In recent years, benefiting from the development of large language models (transformer-based models [13] ) and increased annotation data in the clinical domain, statistical approaches were developed rapidly and have achieved remarkable performance on various clinical NLP tasks. Nonetheless, a substantial gap exists between the performance of these large language models and the understanding of their generalizability [14] . The successfully deployed clinical NLP systems are often developed based on data from a single healthcare institute, their performances can be various if applied in different healthcare institutes [15] .…”

Section: Introductionmentioning

confidence: 99%

“…These studies primarily focused on assessing the generalizability of rule-based and traditional machine learning methods. However, there is currently a dearth of studies investigating the generalizability of transformer-based models in the clinical domain [14] . Khambete et al explored the generalization of the SciBERT model on a clinical sentiment classification task.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records

Zhou,

Wang,

Wang

et al. 2023

Computational and Structural Biotechnology Journal

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records

Zhou,

Wang,

Wang

et al. 2023

Computational and Structural Biotechnology Journal

View full text Add to dashboard Cite

“…[ 28,64]. In order to reduce the cost of labeling, previous studies utilize distant supervision based on the knowledge graph to generate labeled examples automatically [69,74].…”

Section: Relation Extraction Relation Extraction Aims To Identify The...mentioning

confidence: 99%

Ontology-enhanced Prompt-tuning for Few-shot Learning

Ye,

Zhang,

Deng

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Few-shot Learning (FSL) is aimed to make predictions based on a limited number of samples. Structured data such as knowledge graphs and ontology libraries has been leveraged to benefit the few-shot setting in various tasks. However, the priors adopted by the existing methods suffer from challenging knowledge missing, knowledge noise, and knowledge heterogeneity, which hinder the performance for few-shot learning. In this study, we explore knowledge injection for FSL with pre-trained language models and propose ontology-enhanced prompt-tuning (OntoPrompt). Specifically, we develop the ontology transformation based on the external knowledge graph to address the knowledge missing issue, which fulfills and converts structure knowledge to text. We further introduce span-sensitive knowledge injection via a visible matrix to select informative knowledge to handle the knowledge noise issue. To bridge the gap between knowledge and text, we propose a collective training algorithm to optimize representations jointly. We evaluate our proposed OntoPrompt in three tasks, including relation extraction, event extraction, and knowledge graph completion, with eight datasets. Experimental results demonstrate that our approach can obtain better few-shot performance than baselines.

show abstract

“…We acknowledge that some modules of our model depend on pre-trained models such as word embeddings. These models are known to reproduce and even magnify societal bias present in their original training data (Li et al, 2021).…”

Section: Ethical Considerationsmentioning

confidence: 99%

Modeling Intensification for Sign Language Generation: A Computational Approach

İnan¹,

Zhong²,

Hassan³

et al. 2022

Preprint

View full text Add to dashboard Cite

End-to-end sign language generation models do not accurately represent the prosody in sign language. A lack of temporal and spatial variations leads to poor-quality generated presentations that confuse human interpreters. In this paper, we aim to improve the prosody in generated sign languages by modeling intensification in a data-driven manner. We present different strategies grounded in linguistics of sign language that inform how intensity modifiers can be represented in gloss annotations. To employ our strategies, we first annotate a subset of the benchmark PHOENIX-14T, a German Sign Language dataset, with different levels of intensification. We then use a supervised intensity tagger to extend the annotated dataset and obtain labels for the remaining portion of it. This enhanced dataset is then used to train state-of-the-art transformer models for sign language generation. We find that our efforts in intensification modeling yield better results when evaluated with automatic metrics. Human evaluation also indicates a higher preference of the videos generated using our model.

show abstract

On Robustness and Bias Analysis of BERT-Based Relation Extraction

Cited by 7 publications

References 40 publications

A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records

A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records

Ontology-enhanced Prompt-tuning for Few-shot Learning

Modeling Intensification for Sign Language Generation: A Computational Approach

Contact Info

Product

Resources

About