Diagnosing Gender Bias in Image Recognition Systems

Abstract

Image recognition systems offer the promise to learn from images at scale without requiring expert knowledge. However, past research suggests that machine learning systems often produce biased output. In this article, we evaluate potential gender biases of commercial image recognition platforms using photographs of U.S. members of Congress and a large number of Twitter images posted by these politicians. Our crowdsourced validation shows that commercial image recognition systems can produce labels that are correct and biased at the same time as they selectively report a subset of many possible true labels. We find that images of women received three times more annotations related to physical appearance. Moreover, women in images are recognized at substantially lower rates in comparison with men. We discuss how encoded biases such as these affect the visibility of women, reinforce harmful gender stereotypes, and limit the validity of the insights that can be gathered from such data.

Gender, Candidate Emotional Expression, and Voter Reactions During Televised Debates

“…The measures of emotion used in our analyses all have the potential to be biased in their evaluations of the behavior of men and women. As machine learning systems get closer to replicating human behavior, they also replicate human biases (Schwemmer et al 2020). We recognize that these biases may have important theoretical and practical implications for our research.…”

“…Facial displays of emotion: Emotion-detection APIs have a number of biases (including gender and racial biases) encoded in to their processes (Buolamwini and Gebru 2018). For example, Schwemmer et al (2020) find that classifiers are much more likely to assign terms associated with physical appearance to images of female (versus male) members of Congress. There are also gender biases in the classification of specific emotions: a neutral face, happiness, and anger tend to produce the lowest levels of gender bias (Khanal et al 2018).…”

Bridging the computational and visual turn: Re-tooling visual studies with image recognition and network analysis to study online climate images

“…This does not mean we should blindly trust the algorithm to correctly annotate all images. Even though Google's AI has been tested to be more accurate than others (Oberoi, 2016), it has also appropriately been criticised for having racial biases, such as annotating Black people as "gorillas" (Simonite, 2018) or being systematically biased on gender (Schwemmer et al, 2020). The confidence score somewhat improves reliability by giving us a mechanism through which to rule out the most uncertain coding by the AI.…”

Advances in Bias and Fairness in Information Retrieval

“…To detect race and gender bias in search outputs, we relied on their manual coding. While some earlier studies (e.g., [40]) use image recognition for extracting image features, its applicability for bias detection has been questioned recently [45] considering the possibility of recognition approaches being biased themselves. Hence, we used two coders to classify all the collected images based on categories listed below.…”