To "See" is to Stereotype

Abstract

Machine-learned computer vision algorithms for tagging images are increasingly used by developers and researchers, having become popularized as easy-to-use "cognitive services." Yet these tools struggle with gender recognition, particularly when processing images of women, people of color and non-binary individuals. Socio-technical researchers have cited data bias as a key problem; training datasets often over-represent images of people and contexts that convey social stereotypes. The social psychology literature explains that people learn social stereotypes, in part, by observing others in particular roles and contexts, and can inadvertently learn to associate gender with scenes, occupations and activities. Thus, we study the extent to which image tagging algorithms mimic this phenomenon. We design a controlled experiment, to examine the interdependence between algorithmic recognition of context and the depicted person's gender. In the spirit of auditing to understand machine behaviors, we create a highly controlled dataset of people images, imposed on gender-stereotyped backgrounds. Our methodology is reproducible and our code publicly available. Evaluating five proprietary algorithms, we find that in three, gender inference is hindered when a background is introduced. Of the two that "see" both backgrounds and gender, it is the one whose output is most consistent with human stereotyping processes that is superior in recognizing gender. We discuss the accuracy--fairness trade-off, as well as the importance of auditing black boxes in better understanding this double-edged sword.

Algorithmic injustice: a relational ethics approach

“…However, one thing remains constant: individuals and communities that are at the margins of society are disproportionally impacted. Some examples include object detection, 44 search engine results, 45 recidivism, 46 gender recognition, 47 gender classification, 48,49 and medicine. 1 The findings of Wilson et.al., 44 for instance, demonstrate that object detection systems designed to predict pedestrians display higher error rates identifying dark-skinned pedestrians while light-skinned pedestrians are identified with higher precision.…”

Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI

“…Early work on the 2016 U.S. Presidential Election mentioned LGBTQ+ rights as a contentious issue, following the advancement of marriage equality and the repeal of both the U.S. Defense of Marriage Act 16 and the U.S. Military's Don't Ask Don't Tell policy 17 during the Obama administration between 2008 and 2016 [132,216]. This is part of a broader trend we observe of growing attention paid to online political discourse, with LGBTQ+ rights being one of the topics of debate [39,42,111,224].…”

“…It provided states the ability to refuse to recognize same-sex marriages performed in other states. 17 Instituted in 1996, this policy prohibited U.S. Military personnel from discriminating against or harassing closeted or not-publicly out homosexual or bisexual service members. It also barred openly gay, lesbian, or bisexual people from military service.…”

The Impossibility of Automating Ambiguity

“…Hiring tools tend to disproportionately disadvantage women (Ajunwa et al, 2016). Additionally, the notion of gender that ML systems depend on is a fundamentally essentialist one that operationalizes gender in a trans-exclusive way resulting in disproportionate harm to trans people (Barlas et al, 2021;Hamidi et al, 2018;Keyes, 2018). Machine classification and prediction, thus, negatively impact individuals and groups at the margins the most.…”