The sins of the parents are to be laid upon the children: biased humans, biased data, biased models

Abstract

Technological innovations have become a key driver of societal advancements. Nowhere is this more evident than in the field of machine learning (ML), which has developed algorithmic models that shape our decisions, behaviors, and outcomes. These tools have widespread use, in part, because they can synthesize massive amounts of data to make seemingly objective recommendations. Yet, in the past few years, the ML community has been raising the alarm on why we should be cautious in interpreting and using these models: they are created by humans, from data generated by humans, whose psychology allows for various biases that impact how the models are developed, trained, tested and interpreted. As psychologists, we thus face a fork in the road; Down the first path, we can continue to use these models without examining and addressing these critical flaws, and rely on computer scientists to try to mitigate them. Down the second path, we can turn our expertise in bias towards this growing field, collaborating with computer scientists to mitigate the deleterious outcomes associated with these models. This paper serves to light the way down the second path by identifying how extant psychological research can help examine and mitigate bias in ML models.

The Manager in the Machine: Organizational Priorities Influence AI Programmer’s Ability to Design Fair Models

“…Our framework is both theoretically and practically motivated. In terms of the algorithmic fairness literature, this research is theoretically motivated by two sources: first, it is motivated by psychologists' and computer scientists' invitations to management scholars for their input on the growing body of research on fairness in machine learning (Osborne et al 2023). Second, this research is motivated by the growing prevalence (and power) of AI models in organizations.…”

“…Primarily, this research has argued that programmers struggle to recognize that there is bias in AI models. For instance: Osborne et al (2023) note that fairness in AI is a relatively nascent field, and many programmers may not have received training about creating fair models or know that they should look for bias in their models. Indeed, many programmers may not know specifically how bias is input into the model or where to look for it within the model.…”

“…Yet, their solutions to this problem have primarily focused on tweaking the factors and datasets that compose the algorithm (e.g., Chouldechova and Roth 2018). Recently, social scientists (e.g., Benjamin 2019, Osborne et al 2023, Rambachan et al 2020) have begun contributing solutions of their own -looking to highlight how factors such as government regulation, society-level stereotyping, and programmer-level decision-making influence algorithmic bias. While these contributions have been useful, they oftentimes do not account for an important consideration: AI models, frequently, are constructed by programmers who are embedded within organizations.…”

“…However, for programmers to be able to address bias, three things need to happen: first, they must be aware of how bias can manifest in algorithmic output; which is an important hindrance for many programmers (Osborne et al 2023). Second, provided they can identify bias in the model, they must recognize that there is an opportunity to address it.…”

Diversity, equity, and inclusivity in observational ambulatory assessment: Recommendations from two decades of Electronically Activated Recorder (EAR) research

“…In spite of the resource-saving potential, we similarly urge caution and careful psychometric consideration. Decades of research in social psychology have established that judgments of human behavior are strongly subject to prejudice and bias (Greenwald & Krieger, 2006), and lessons from NLP suggest that these biases will inevitably permeate early attempts at behavioral coding algorithms (Osborne et al, 2022). In the long run, many of our recommendations for mitigating sources of biases in human coders (see "Coding System Development" and "Recruiting and Training Coders") may be adapted for the training of machine learning algorithms.…”