Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance

Garnerin, Mahault; Rossato, Solange; Besacier, Laurent

doi:10.1145/3347449.3357480

Cited by 25 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Pioneer work from (Adda-Decker and Lamel, 2005) found better performance on women's voices, while a preliminary research on YouTube automatic caption system found better recognition rate of male speech but no gender-difference in a follow-up study (Tatman and Kasten, 2017). Recent work on hybrid ASR systems observed that gender imbalance in data could lead to decreased ASR performance on the gender category least represented (Garnerin et al, 2019). This last study was conducted on French broadcast data in which women account for only 35% of the speakers.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing

2021

View full text Add to dashboard Cite

Languages differ in terms of the absence or presence of gender features, the number of gender classes and whether and where gender features are explicitly marked. These cross-linguistic differences can lead to ambiguities that are difficult to resolve, especially for sentence-level MT systems. The identification of ambiguity and its subsequent resolution is a challenging task for which currently there aren't any specific resources or challenge sets available. In this paper, we introduce gENder-IT, an English-Italian challenge set focusing on the resolution of natural gender phenomena by providing word-level gender tags on the English source side and multiple gender alternative translations, where needed, on the Italian target side.

show abstract

Section: Introductionmentioning

confidence: 99%

Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing

2021

View full text Add to dashboard Cite

show abstract

“…Pioneer work from (Adda-Decker and Lamel, 2005) found better performance on women's voices, while a preliminary research on YouTube automatic caption system found better recognition rate of male speech (Tatman, 2017) but no gender-difference in a follow-up study (Tatman and Kasten, 2017). Recent work on hybrid ASR systems observed that gender imbalance in data could lead to decreased ASR performance on the gender category least represented (Garnerin et al, 2019). This last study was conducted on French broadcast data in which women account for only 35% of the speakers.…”

Section: Introductionmentioning

confidence: 99%

Investigating the Impact of Gender Representation in ASR Training Data: a Case Study on Librispeech

Garnerin¹,

Rossato²,

Besacier³

2021

Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

In this paper we question the impact of gender representation in training data on the performance of an end-to-end ASR system. We create an experiment based on the Librispeech corpus and build 3 different training corpora varying only the proportion of data produced by each gender category. We observe that if our system is overall robust to the gender balance or imbalance in training data, it is nonetheless dependant of the adequacy between the individuals present in the training and testing sets.

show abstract

“…For instance, ASR systems have been shown to struggle with speech variance due to gender, age, speech impairment, race, and accents. Several studies on different languages have found gender differences: although most studies report that female speech is recognised better than male speech (Arabic [1], English [2][3][4], and French [3]), the reverse pattern is also found (French [5], English [6]), although no difference in the recognition of male and female speech was found in a follow-up study of the latter study [7] nor was a difference found in [5]. [1] found that speakers younger than 30 years of age were better recognised than those older than 30 years.…”

Section: Introductionmentioning

confidence: 99%

Quantifying Bias in Automatic Speech Recognition

Feng¹,

Kudina²,

Halpern³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automatic speech recognition (ASR) systems promise to deliver objective interpretation of human speech. Practice and recent evidence suggests that the state-of-the-art (SotA) ASRs struggle with the large variation in speech due to e.g., gender, age, speech impairment, race, and accents. Many factors can cause the bias of an ASR system. Our overarching goal is to uncover bias in ASR systems to work towards proactive bias mitigation in ASR. This paper is a first step towards this goal and systematically quantifies the bias of a Dutch SotA ASR system against gender, age, regional accents and non-native accents. Word error rates are compared, and an in-depth phoneme-level error analysis is conducted to understand where bias is occurring. We primarily focus on bias due to articulation differences in the dataset. Based on our findings, we suggest bias mitigation strategies for ASR development.

show abstract

Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance

Cited by 25 publications

References 19 publications

Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing

Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing

Investigating the Impact of Gender Representation in ASR Training Data: a Case Study on Librispeech

Quantifying Bias in Automatic Speech Recognition

Contact Info

Product

Resources

About