Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns

Webster, Kellie; Recasens, Marta Vilar; Axelrod, Vera; Baldridge, Jason

doi:10.1162/tacl_a_00240

Cited by 210 publications

(277 citation statements)

References 31 publications

Supporting

Mentioning

259

Contrasting

Order By: Relevance

“…We evaluate our BERT-based models on two benchmarks: the paragraph-level GAP dataset (Webster et al, 2018), and the documentlevel English OntoNotes 5.0 dataset (Pradhan et al, 2012). OntoNotes examples are considerably longer and typically require multiple segments to read the entire document.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

BERT for Coreference Resolution: Baselines and Analysis

Joshi

Levy

Zettlemoyer

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

246

278

View full text Add to dashboard Cite

We apply BERT to coreference resolution, achieving strong improvements on the OntoNotes (+3.9 F1) and GAP (+11.5 F1) benchmarks. A qualitative analysis of model predictions indicates that, compared to ELMo and BERT-base, BERT-large is particularly better at distinguishing between related but distinct entities (e.g., President and CEO). However, there is still room for improvement in modeling document-level context, conversations, and mention paraphrasing. Our code and models are publicly available 1 .

show abstract

Section: Methodsmentioning

confidence: 99%

“…We fine-tune BERT to coreference resolution, achieving strong improvements on the GAP (Webster et al, 2018) and OntoNotes (Pradhan et al, 2012) benchmarks. We present two ways of extending the c2f-coref model in .…”

Section: Introductionmentioning

confidence: 99%

BERT for Coreference Resolution: Baselines and Analysis

Joshi

Levy

Zettlemoyer

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

246

278

View full text Add to dashboard Cite

show abstract

“…Another GBET for coreference resolution named GAP contains sentences mined from Wikipedia and thus can perform an evaluation with sentences taken from real contexts as opposed to artificially generated ones (Webster et al, 2018). GAP does not include stereotypical nouns; instead, pronouns refer to names only.…”

Section: Taskmentioning

confidence: 99%

Mitigating Gender Bias in Natural Language Processing: Literature Review

Sun¹,

Gaut²,

Tang³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

317

228

View full text Add to dashboard Cite

As Natural Language Processing (NLP) and Machine Learning (ML) tools rise in popularity, it becomes increasingly vital to recognize the role they play in shaping societal biases and stereotypes. Although NLP models have shown success in modeling various applications, they propagate and may even amplify gender bias found in text corpora. While the study of bias in artificial intelligence is not new, methods to mitigate gender bias in NLP are relatively nascent. In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP. We discuss gender bias based on four forms of representation bias and analyze methods recognizing gender bias. Furthermore, we discuss the advantages and drawbacks of existing gender debiasing methods. Finally, we discuss future studies for recognizing and mitigating gender bias in NLP.

show abstract

“…The GAP Coreference Dataset 3 (Webster et al, 2018) has 4454 records and officially split into three parts: development set (2000 records), test set (2000 records), and validation set (454 records). Conforming to the stage 1 of Gendered Pronoun Resolution 4 task, the official test set and validation set are combined as the training dataset in the experiments, while the official development set is used as the test set correspondingly.…”

Section: Datasetmentioning

confidence: 99%

MSnet: A BERT-based Network for Gendered Pronoun Resolution

Wang¹

2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

View full text Add to dashboard Cite

The pre-trained BERT model achieves a remarkable state of the art across a wide range of tasks in natural language processing. For solving the gender bias in gendered pronoun resolution task, I propose a novel neural network model based on the pre-trained BERT. This model is a type of mention score classifier and uses an attention mechanism with no parameters to compute the contextual representation of entity span, and a vector to represent the triple-wise semantic similarity among the pronoun and the entities. In stage 1 of the gendered pronoun resolution task, a variant of this model, trained in the fine-tuning approach, reduced the multi-class logarithmic loss to 0.3033 in the 5-fold cross-validation of training set and 0.2795 in testing set. Besides, this variant won the 2nd place with a score at 0.17289 in stage 2 of the task.The code in this paper is available at: https://github.com/ziliwang/ MSnet-for-Gendered-Pronoun-Resolution

show abstract

Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns

Cited by 210 publications

References 31 publications

BERT for Coreference Resolution: Baselines and Analysis

BERT for Coreference Resolution: Baselines and Analysis

Mitigating Gender Bias in Natural Language Processing: Literature Review

MSnet: A BERT-based Network for Gendered Pronoun Resolution

Contact Info

Product

Resources

About