Robots Enact Malignant Stereotypes

Abstract

Stereotypes, bias, and discrimination have been extensively documented in Machine Learning (ML) methods such as Computer Vision (CV) [18,80], Natural Language Processing (NLP) [6], or both, in the case of large image and caption models such as OpenAI CLIP [14].In this paper, we evaluate how ML bias manifests in robots that physically and autonomously act within the world. We audit one of several recently published CLIP-powered robotic manipulation methods, presenting it with objects that have pictures of human faces on the surface which vary across race and gender, alongside task descriptions that contain terms associated with common stereotypes.Our experiments definitively show robots acting out toxic stereotypes with respect to gender, race, and scientifically-discredited physiognomy, at scale. Furthermore, the audited methods are less likely to recognize Women and People of Color. Our interdisciplinary sociotechnical analysis synthesizes across fields and applications such as Science Technology and Society (STS), Critical Studies, History, Safety, Robotics, and AI. We find that robots powered by large datasets and Dissolution Models (sometimes called "foundation models", e.g. CLIP) that contain humans risk physically amplifying malignant stereotypes in general; and that merely correcting disparities will be insufficient for the complexity and scale of the problem. Instead, we recommend that robot learning methods that physically manifest stereotypes or other harmful outcomes be paused, reworked, or even wound down when appropriate, until outcomes can be proven safe, effective, and just. Finally, we discuss comprehensive policy changes and the potential of new interdisciplinary research on topics like Identity Safety Assessment Frameworks and Design Justice to better understand and address these harms.

ELT in the Age of Artificial Intelligence (AI): Working with Machines

“…Similarly, Hundt et al (2022) reported that many training datasets have demonstrated spew racism, sexism, and other detrimental biases. They called for measures and regulations governing the design and training of robots and other AI systems.…”

Guidelines for Robotics

“…Social hierarchy Like data-driven algorithms, robots are beginning to show behaviors reproducing malicious stereotypes [12] existing in the social world (racism, patriarchy, etc.). These behaviors, reproducing the logics coming from authoritarian structures of the social world, which is largely underestimated, must become a major problem of robotics.…”

“…1) produced by robots with ethical and socio-political impacts. Initial experiments are underway [12], but this area of research remains largely under-investigated.…”

Evaluating the Impact and Potential Impact of Machine Learning on Medical Decision Making

“…[6][7][8] Famously, ML-driven tools mirror the structural racism and cultural bias of the system from which the data are sourced. [9][10][11] Further, in the automated model selection process, the overall quality of model prediction may be achieved by sacrificing the quality of prediction for marginalized persons and other underrepresented minority groups in exchange for superior quality of prediction for majority groups. 12 Ultimately, the use of an ML tool to direct decisions may create homogeneity in decisions among similar patients, and as those patients become part of the data set for future predictions, they may enrich the data set in unbalanced and potentially self-fulfilling ways.…”

“… 6 – 8 Famously, ML-driven tools mirror the structural racism and cultural bias of the system from which the data are sourced. 9 – 11 Further, in the automated model selection process, the overall quality of model prediction may be achieved by sacrificing the quality of prediction for marginalized persons and other underrepresented minority groups in exchange for superior quality of prediction for majority groups. 12 …”