Moving beyond “algorithmic bias is a data problem”

Hooker, Sara

doi:10.1016/j.patter.2021.100241

Cited by 116 publications

(74 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Their focus is bias in autonomous systems such as self-driving cars, and offer the example of using a biased estimator to minimize variance if you have a small sample size. Another exception is Hooker [ 24 ], who makes a plea for looking beyond just data bias, arguing that model design also contributes to bias, citing examples from work in facial recognition.…”

Section: Data People or Algorithms?mentioning

confidence: 99%

Algorithms are not neutral

Stinson

2022

AI Ethics

View full text Add to dashboard Cite

When Artificial Intelligence (AI) is applied in decision-making that affects people’s lives, it is now well established that the outcomes can be biased or discriminatory. The question of whether algorithms themselves can be among the sources of bias has been the subject of recent debate among Artificial Intelligence researchers, and scholars who study the social impact of technology. There has been a tendency to focus on examples, where the data set used to train the AI is biased, and denial on the part of some researchers that algorithms can also be biased. Here we illustrate the point that algorithms themselves can be the source of bias with the example of collaborative filtering algorithms for recommendation and search. These algorithms are known to suffer from cold-start, popularity, and homogenizing biases, among others. While these are typically described as statistical biases rather than biases of moral import; in this paper we show that these statistical biases can lead directly to discriminatory outcomes. The intuitive idea is that data points on the margins of distributions of human data tend to correspond to marginalized people. The statistical biases described here have the effect of further marginalizing the already marginal. Biased algorithms for applications such as media recommendations can have significant impact on individuals’ and communities’ access to information and culturally-relevant resources. This source of bias warrants serious attention given the ubiquity of algorithmic decision-making.

show abstract

Section: Data People or Algorithms?mentioning

confidence: 99%

Algorithms are not neutral

Stinson

2022

AI Ethics

View full text Add to dashboard Cite

show abstract

“…Learning bias arises when modeling choices amplify performance disparities across different examples in the data. 18 For example, an important modeling choice is the objective function that an ML algorithm learns to optimize during training. Typically, these functions encode some measure of accuracy on the task (e.g., cross-entropy loss for classification problems or mean squared error for regression problems).…”

Section: Learning Biasmentioning

confidence: 99%

Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle

Suresh¹,

Guttag²

2021

MIT Case Studies in Social and Ethical Responsibilities of Computing

View full text Add to dashboard Cite

As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this case study, we provide a framework that identifies seven distinct potential sources of downstream harm in machine learning, spanning data collection, development, and deployment. We describe how these issues arise, how they are relevant to particular applications, and how they motivate different mitigations.

show abstract

“…Thus, on many occasions, a perfectly balanced dataset cannot be obtained and algorithmic solutions may come in handy. As discussed in a recent article [ 4 ], we need to move beyond the idea that “algorithmic bias is a data problem” and start acknowledging that algorithms are not impartial, and some design choices are better than others. In that sense, the choice of specific model architectures, loss functions, and training strategies plays a fundamental role in amplifying or mitigating potential equity issues because they are meant to induce specific behaviour in our systems.…”

Section: It Is Not All About Datamentioning

confidence: 99%

“…Recently, the research community of fairness in ML has shown that, contrary to popular belief about computer systems, these models can be far from objective and the decisions that they make can be strongly influenced—even unintentionally—by population demographic factors such as sex, ethnicity, or age, resulting in poor performance for specific subgroups [ 1 , 2 ]. The causes behind this phenomenon are multiple, and range from lack of diversity in the team of developers [ 3 ] to the technical design choices in terms of model architecture, objective functions, and training algorithms [ 4 ]. Another fundamental aspect is the data used to train these models.…”

Section: Introductionmentioning

confidence: 99%

On the relationship between research parasites and fairness in machine learning: challenges and opportunities

et al. 2021

View full text Add to dashboard Cite

Machine learning systems influence our daily lives in many different ways. Hence, it is crucial to ensure that the decisions and recommendations made by these systems are fair, equitable, and free of unintended biases. Over the past few years, the field of fairness in machine learning has grown rapidly, investigating how, when, and why these models capture, and even potentiate, biases that are deeply rooted not only in the training data but also in our society. In this Commentary, we discuss challenges and opportunities for rigorous posterior analyses of publicly available data to build fair and equitable machine learning systems, focusing on the importance of training data, model construction, and diversity in the team of developers. The thoughts presented here have grown out of the work we did, which resulted in our winning the annual Research Parasite Award that GigaSciencesponsors.

show abstract

Moving beyond “algorithmic bias is a data problem”

Cited by 116 publications

References 5 publications

Algorithms are not neutral

Algorithms are not neutral

Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle

On the relationship between research parasites and fairness in machine learning: challenges and opportunities

Contact Info

Product

Resources

About