One Label, One Billion Faces

Abstract

Computer vision is widely deployed, has highly visible, societyaltering applications, and documented problems with bias and representation. Datasets are critical for benchmarking progress in fair computer vision, and often employ broad racial categories as population groups for measuring group fairness. Similarly, diversity is often measured in computer vision datasets by ascribing and counting categorical race labels. However, racial categories are ill-defined, unstable temporally and geographically, and have a problematic history of scientific use. Although the racial categories used across datasets are superficially similar, the complexity of human race perception suggests the racial system encoded by one dataset may be substantially inconsistent with another. Using the insight that a classifier can learn the racial system encoded by a dataset, we conduct an empirical study of computer vision datasets supplying categorical race labels for face images to determine the cross-dataset consistency and generalization of racial categories. We find that each dataset encodes a substantially unique racial system, despite nominally equivalent racial categories, and some racial categories are systemically less consistent than others across datasets. We find evidence that racial categories encode stereotypes, and exclude ethnic groups from categories on the basis of nonconformity to stereotypes. Representing a billion humans under one racial category may obscure disparities and create new ones by encoding stereotypes of racial systems. The difficulty of adequately converting the abstract concept of race into a tool for measuring fairness underscores the need for a method more flexible and culturally aware than racial categories. CCS CONCEPTS • Social and professional topics → Race and ethnicity; • Computing methodologies → Computer vision.

Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"

“…These services, generally available 'off-the-shelf' to anyone, aim to determine an individual's facial characteristics including physical or demographic traits based on an image of their face. However, human faces are not a homogeneous group [75]; the set of images used to train and evaluate the underlying model will have a significant influence over the model's behaviour and reported performance. Without knowing specifically how and where their customers will use their services, the AI API providers are at constant risk of failing to envision, let alone account for, all the various contexts their models' might encounter.…”

An Empirical Analysis of Racial Categories in the Algorithmic Fairness Literature

“…Scientists may choose a given racial classification for a variety of reasons, including widespread acceptance, the ability to facilitate comparisons across studies, and stability [40]. Inconsistencies in racial categories have been noted in many disciplines including survey methods [36], public health [18,26], and computer vision [19,39]. Although differences in racial classification can affect research conclusions [36], researchers often fail to explain or justify their operationalizations of race [23,39].…”

Constructing categories: Moving beyond protected classes in algorithmic fairness

“…Databases used for training facial recognition algorithms contain assumptions about the static and apolitical nature of these categories, but rarely explain how their categories were constructed (Scheuerman et al, 2020). These databases also often use labels generated by outsourced human labor like Amazon Mechanical Turk, which rely on human assumptions drawn from visual images alone, and only designate a small number of racial categories (Khan & Fu, 2021). The policies used when creating datasets therefore have lasting impacts on the categories to which we have access, the unfairness we can measure, and the policy changes we recommend based on model outcomes (Kasy & Abebe, 2021).…”