Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts

Merullo, Jack; Yeh, Luke; Handler, Abram; Grissom, Alvin; O’Connor, Brendan; Iyyer, Mohit

doi:10.18653/v1/d19-1666

Cited by 16 publications

(14 citation statements)

References 22 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…included in 'Social Science/Social Media' in Table 1) uses NLP tools to analyze race in society. Examples include examining how commentators describe football players of different races (Merullo et al, 2019) or how words like 'prejudice' have changed meaning over time (Vylomova et al, 2019). While differing in goals, this work is often susceptible to the same pitfalls as other NLP tasks.…”

Section: Social Analyses Of Outputsmentioning

confidence: 99%

A Survey of Race, Racism, and Anti-Racism in NLP

Field¹,

Blodgett²,

Waseem³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Despite inextricable ties between race and language, little work has considered race in NLP research and development. In this work, we survey 79 papers from the ACL anthology that mention race. These papers reveal various types of race-related bias in all stages of NLP model development, highlighting the need for proactive consideration of how NLP systems can uphold racial hierarchies. However, persistent gaps in research on race and NLP remain: race has been siloed as a niche topic and remains ignored in many NLP tasks; most work operationalizes race as a fixed singledimensional variable with a ground-truth label, which risks reinforcing differences produced by historical racism; and the voices of historically marginalized people are nearly absent in NLP literature. By identifying where and how NLP literature has and has not considered race, especially in comparison to related fields, our work calls for inclusion and racial justice in NLP research practices. 3 The ACL Anthology includes papers from all official ACL venues and some non-ACL events listed in Appendix A, as of December 2020 it included 6, 200 papers 5 We note that conceptualizations of AAE and the accompanying terminology for the variety have shifted considerably in the last half century; see King (2020) for an overview.6 https://bit.ly/2Yv07IL 7 https://bit.ly/3j2weZA

show abstract

Section: Social Analyses Of Outputsmentioning

confidence: 99%

A Survey of Race, Racism, and Anti-Racism in NLP

Field¹,

Blodgett²,

Waseem³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Sports Event Datasets and Tasks. Commentary in the sports domain has been collected to study a variety of problems such as racial bias in football game reporting (Merullo et al, 2019) and gender construction in NBA/WNBA coverage (Aull and Brown, 2013). However, these datasets do not provide any information on the temporal alignment between commentary and events.…”

Section: Information Densitymentioning

confidence: 99%

“…Sports matches provide an ideal test bed for state tracking due to their self-contained, fully observable nature and their inherent interpretability in the form of the temporal evolution of scores. However, existing sports-related commentary collections such as described by Aull and Brown (2013) and Merullo et al (2019) do not provide such within-match temporal information.…”

Section: Introductionmentioning

confidence: 99%

SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain

Zhang¹,

Eickhoff

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

In the pursuit of natural language understanding, there has been a long standing interest in tracking state changes throughout narratives. Impressive progress has been made in modeling the state of transaction-centric dialogues and procedural texts. However, this problem has been less intensively studied in the realm of general discourse where ground truth descriptions of states may be loosely defined and state changes are less densely distributed over utterances. This paper proposes to turn to simplified, fully observable systems that show some of these properties: Sports events. We curated 2,263 soccer matches including timestamped natural language commentary accompanied by discrete events such as a team scoring goals, switching players or being penalized with cards. We propose a new task formulation where, given paragraphs of commentary of a game at different timestamps, the system is asked to recognize the occurrence of in-game events. This domain allows for rich descriptions of state while avoiding the complexities of many other real-world settings. As an initial point of performance measurement, we include two baseline methods from the perspectives of sentence classification with temporal dependence and current state-of-the-art generative model, respectively, and demonstrate that even sophisticated existing methods struggle on the state tracking task when the definition of state broadens or non-event chatter becomes prevalent.

show abstract

“…Machine learning algorithms learn to model patterns present in training datasets. In particular, they make predictions that directly reflect the harmful societal biases present in training datasets, such as racial bias in sports reports (Merullo et al, 2019) and political bias in news data . Such biases are rife in NLP, for example, in learned word embeddings (Bolukbasi et al, 2016;Brunet et al, 2018;, visual semantic role labeling (Zhao et al, 2017), natural language inference (He et al, 2019), abusive language classification (Park et al, 2018), and * Joint first authors.…”

Section: Introductionmentioning

confidence: 99%

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

Dinan

Fan

Williams

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

102

View full text Add to dashboard Cite

Social biases present in data are often directly reflected in the predictions of models trained on that data. We analyze gender bias in dialogue data, and examine how this bias is not only replicated, but is also amplified in subsequent generative chit-chat dialogue models. We measure gender bias in six existing dialogue datasets before selecting the most biased one, the multi-player textbased fantasy adventure dataset LIGHT (Urbanek et al., 2019), as a testbed for bias mitigation techniques. We consider three techniques to mitigate gender bias: counterfactual data augmentation, targeted data collection, and bias controlled training. We show that our proposed techniques mitigate gender bias by balancing the genderedness of generated dialogue utterances, and find that they are particularly effective in combination. We evaluate model performance with a variety of quantitative methods-including the quantity of gendered words, a dialogue safety classifier, and human assessments-all of which show that our models generate less gendered, but equally engaging chit-chat responses.

show abstract

Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts

Cited by 16 publications

References 22 publications

A Survey of Race, Racism, and Anti-Racism in NLP

A Survey of Race, Racism, and Anti-Racism in NLP

SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

Contact Info

Product

Resources

About