Bias in data‐driven artificial intelligence systems—An introductory survey

Abstract

Artificial Intelligence (AI)-based systems are widely employed nowadays to make decisions that have far-reaching impact on individuals and society. Their decisions might affect everyone, everywhere, and anytime, entailing concerns about potential human rights issues. Therefore, it is necessary to move beyond traditional AI algorithms optimized for predictive performance and embed ethical and legal principles in their design, training, and deployment to ensure social good while still benefiting from the huge potential of the AI technology. The goal of this survey is to provide a broad multidisciplinary overview of the area of bias in AI systems, focusing on technical challenges and solutions as well as to suggest new research directions towards approaches well-grounded in a legal frame. In this survey, we focus on data-driven AI, as a large part of AI is powered nowadays by (big) data and powerful machine learning algorithms. If otherwise not specified, we use the general term bias to describe problems related to the gathering or processing of data that might result in prejudiced decisions on the bases of demographic features such as race, sex, and so forth.This article is categorized under:

Artificial Intelligence (AI) or Intelligence Augmentation (IA): What Is the Future?

“…Therefore, an ethical framework must be established to set out how AI should be used to prepare for the next-generation of IA. Fairness is dynamic and a social construct and cannot be trusted to automation [80]. By analyzing the characteristics of AI and IA, we have determined that the goal of the human species is IA by making use of AI.…”

Network-based ranking in social systems: three challenges

“…Although we have focused on network-based ranking algorithms, these challenges potentially apply to any ranking method for a social system. This is exemplified by recent cross-disciplinary efforts to understand, quantify, and mitigate the bias of machine-learning algorithms employed by governments and organizations [37], and to predict the long-term consequences of such biases through computer simulations [62].…”

“…However, accepting blindly the outcomes by a ranking algorithm is potentially misleading, and we argue here that an appropriate dose of caution is necessary when interpreting the results by a given algorithm as a signal of quality or talent. An important shortcoming of ranking algorithms is that-in a similar way as machine learning algorithms [37]-they can be biased by multiple confounding factors.…”

Administration by algorithm: A risk management framework

“…What is possibly even more worrying is that some of these problems may not be bugs, but features, i.e. policies and/or prejudices consciously or unconsciously built into the code and/or problems caused by biases in the underlying data that they use (Ntoutsi et al, 2020).…”