Syntactic and cognitive issues in investigating gendered coreference

The pronoun “they” can be either plural or singular, perhaps referring to an individual who identifies as nonbinary. How do listeners identify whether “they” has a singular or plural sense? We test the role of explicitly discussing pronouns (e.g., “Alex uses they/them pronouns”). In three experiments, participants read short stories, like “Alex went running with Liz. They fell down.” Answers to “Who fell down” indicated whether participants interpreted they as Alex or Alex-and-Liz. We found more singular responses in discourse contexts that make Alex more available: when Alex was either the only person in the context or mentioned first. Critically, the singular interpretation was stronger when participants heard explicit instructions that Alex uses they/them pronouns, even though participants in all conditions had ample opportunity to learn this fact through observation. Results show that the social trend to talk about pronouns has a direct impact on how language is understood.

show abstract

“…We couch our interpretation within Ackerman’s ( 2019 ) framework, which proposes three levels of representation of gender (see Fig. 4 ).…”

Section: Discussionmentioning

confidence: 99%

My pronouns are they/them: Talking about pronouns changes how pronouns are understood

2021

View full text Add to dashboard Cite

show abstract

“…As well, some languages have adopted gender-neutral singular pronouns and profession terms, both to include non-binary people and to avoid the social biases of gendered language (Misersky et al, 2019). However, the target languages supported by WinoMT lack widely-accepted non-binary inflection conventions (Ackerman, 2019). This paper addresses gender bias that can be resolved at the sentence level and evaluated with existing test sets, and does not address these broader challenges.…”

Section: Gender Bias In Machine Translationmentioning

confidence: 99%

Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem

Saunders¹,

Byrne²

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. In Neural Machine Translation (NMT) gender bias has been shown to reduce translation quality, particularly when the target language has grammatical gender. The recent WinoMT challenge set allows us to measure this effect directly (Stanovsky et al., 2019).Ideally we would reduce system bias by simply debiasing all data prior to training, but achieving this effectively is itself a challenge. Rather than attempt to create a 'balanced' dataset, we use transfer learning on a small set of trusted, gender-balanced examples. This approach gives strong and consistent improvements in gender debiasing with much less computational cost than training from scratch.A known pitfall of transfer learning on new domains is 'catastrophic forgetting', which we address both in adaptation and in inference. During adaptation we show that Elastic Weight Consolidation allows a performance trade-off between general translation quality and bias reduction. During inference we propose a latticerescoring scheme which outperforms all systems evaluated in Stanovsky et al. (2019) on WinoMT with no degradation of general test set BLEU, and we show this scheme can be applied to remove gender bias in the output of 'black box' online commercial MT systems. We demonstrate our approach translating from English into three languages with varied linguistic properties and data availability.

show abstract

“…The Merriam-Webster dictionary recognised 'they' as the word of the year in 2019, noting that it has been deployed as a singular pronoun since at least the late 1300 s. Accordingly, this article focuses on translation into richly gender-inflected languages (e.g., German) and the experimental set-up is based on the recent NMT gender bias evaluation framework WinoMT (Stanovsky et al 2019). While many inflected languages clearly demonstrate the effects of male-female imbalances in training data for the translation of (say) pronouns and nouns, they often lack widely accepted non-binary inflection conventions (Ackerman 2019). Consequently, the options for analysing NMT gender bias in relation to anything other than binary linguistic inflections are currently severely limited.…”

Section: Gender Bias In Nmt Systemsmentioning

confidence: 99%

The practical ethics of bias reduction in machine translation: why domain adaptation is better than data debiasing

et al. 2021

View full text Add to dashboard Cite

This article probes the practical ethical implications of AI system design by reconsidering the important topic of bias in the datasets used to train autonomous intelligent systems. The discussion draws on recent work concerning behaviour-guiding technologies, and it adopts a cautious form of technological utopianism by assuming it is potentially beneficial for society at large if AI systems are designed to be comparatively free from the biases that characterise human behaviour. However, the argument presented here critiques the common well-intentioned requirement that, in order to achieve this, all such datasets must be debiased prior to training. By focusing specifically on gender-bias in Neural Machine Translation (NMT) systems, three automated strategies for the removal of bias are considered – downsampling, upsampling, and counterfactual augmentation – and it is shown that systems trained on datasets debiased using these approaches all achieve general translation performance that is much worse than a baseline system. In addition, most of them also achieve worse performance in relation to metrics that quantify the degree of gender bias in the system outputs. By contrast, it is shown that the technique of domain adaptation can be effectively deployed to debias existing NMT systems after they have been fully trained. This enables them to produce translations that are quantitatively far less biased when analysed using gender-based metrics, but which also achieve state-of-the-art general performance. It is hoped that the discussion presented here will reinvigorate ongoing debates about how and why bias can be most effectively reduced in state-of-the-art AI systems.

show abstract

Syntactic and cognitive issues in investigating gendered coreference

Cited by 55 publications

References 72 publications

My pronouns are they/them: Talking about pronouns changes how pronouns are understood

My pronouns are they/them: Talking about pronouns changes how pronouns are understood

Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem

The practical ethics of bias reduction in machine translation: why domain adaptation is better than data debiasing

Contact Info

Product

Resources

About