The Gender Gap Tracker: Using Natural Language Processing to measure gender bias in media

Abstract

We examine gender bias in media by tallying the number of men and women quoted in news text, using the Gender Gap Tracker, a software system we developed specifically for this purpose. The Gender Gap Tracker downloads and analyzes the online daily publication of seven English-language Canadian news outlets and enhances the data with multiple layers of linguistic information. We describe the Natural Language Processing technology behind this system, the curation of off-the-shelf tools and resources that we used to build it, and the parts that we developed. We evaluate the system in each language processing task and report errors using real-world examples. Finally, by applying the Tracker to the data, we provide valuable insights about the proportion of people mentioned and quoted, by gender, news organization, and author gender. Data collected between October 1, 2018 and September 30, 2020 shows that, in general, men are quoted about three times as frequently as women. While this proportion varies across news outlets and time intervals, the general pattern is consistent. We believe that, in a world with about 50% women, this should not be the case. Although journalists naturally need to quote newsmakers who are men, they also have a certain amount of control over who they approach as sources. The Gender Gap Tracker relies on the same principles as fitness or goal-setting trackers: By quantifying and measuring regular progress, we hope to motivate news organizations to provide a more diverse set of voices in their reporting.

Gender Bias in the News: A Scalable Topic Modelling and Visualization Framework

“…However, research on sports reporting shows that this traditional gendering can be countered, especially when more female reporters are assigned to cover sports events ( Kian and Hardin, 2009 ). We have found in our previous work that female writers are more likely to quote women ( Asr et al, 2021 ), but more extensive analyses suggest that there is no relationship between the proportion of women producing the news and the proportion of women featured in the news ( Kassova, 2020b ). It is possible that the culture in newsrooms is male dominated, thus resulting in professional identity being stronger than gender identity ( Hanitzsch and Hanusch, 2012 ; Kassova, 2020b ).…”

“…Data was obtained for the Gender Gap Tracker and analyzed for this related project. The Gender Gap Tracker data is scraped daily from the websites of seven major Canadian news outlets in English ( Asr et al, 2021 ). We built scrapers from scratch for each outlet and maintain them regularly, as the sites change in structure frequently.…”

“…Using the Gender Gap Tracker’s existing Natural Language Processing pipeline ( Asr et al, 2021 ), the data is enriched with information about the people mentioned (via Named Entity Recognition) and the people quoted, whom we refer to as sources . For the latter, we first identify quotes (reported speech, whether direct or indirect speech) in the text and then associate a speaker with them, using coreference resolution.…”

“…Numbers alone tell a compelling story about the lack of female voices in the media, as we show in more detailed analyses of the Gender Gap Tracker data (Asr et al, 2021). Our goal in this paper is to explore how certain news topics contribute disproportionately to those percentages, and whether we can identify any systemic patterns in the language used in articles where each gender dominates, i.e., in articles with majority male or female sources.…”

“…Our analyses rely on statistics provided by the Gender Gap Tracker, 1 a purpose-built tool that measures the proportion of male and female sources quoted in mainstream Canadian media in English. The Gender Gap Tracker is an automated software system that monitors men and women’s voices on seven major Canadian news outlets in real time, using Natural Language Processing ( Asr et al, 2021 ). The goals of the Tracker are to enhance awareness of women’s portrayal in public discourse through hard evidence and to encourage news organizations to provide a more diverse set of voices in their reporting.…”