Bias and Unfairness in Machine Learning Models: A Systematic Review on Datasets, Tools, Fairness Metrics, and Identification and Mitigation Methods

Abstract

One of the difficulties of artificial intelligence is to ensure that model decisions are fair and free of bias. In research, datasets, metrics, techniques, and tools are applied to detect and mitigate algorithmic unfairness and bias. This study examines the current knowledge on bias and unfairness in machine learning models. The systematic review followed the PRISMA guidelines and is registered on OSF plataform. The search was carried out between 2021 and early 2022 in the Scopus, IEEE Xplore, Web of Science, and Google Scholar knowledge bases and found 128 articles published between 2017 and 2022, of which 45 were chosen based on search string optimization and inclusion and exclusion criteria. We discovered that the majority of retrieved works focus on bias and unfairness identification and mitigation techniques, offering tools, statistical approaches, important metrics, and datasets typically used for bias experiments. In terms of the primary forms of bias, data, algorithm, and user interaction were addressed in connection to the preprocessing, in-processing, and postprocessing mitigation methods. The use of Equalized Odds, Opportunity Equality, and Demographic Parity as primary fairness metrics emphasizes the crucial role of sensitive attributes in mitigating bias. The 25 datasets chosen span a wide range of areas, including criminal justice image enhancement, finance, education, product pricing, and health, with the majority including sensitive attributes. In terms of tools, Aequitas is the most often referenced, yet many of the tools were not employed in empirical experiments. A limitation of current research is the lack of multiclass and multimetric studies, which are found in just a few works and constrain the investigation to binary-focused method. Furthermore, the results indicate that different fairness metrics do not present uniform results for a given use case, and that more research with varied model architectures is necessary to standardize which ones are more appropriate for a given context. We also observed that all research addressed the transparency of the algorithm, or its capacity to explain how decisions are taken.

Evaluation of artificial intelligence techniques in disease diagnosis and prediction

“…These techniques are not mutually exclusive and can be combined to improve the model's performance. The technique selection depends on the data type and the specific problem [ 66 ].…”

Assessment of the bias of artificial intelligence generated images and large language models on their depiction of a surgeon

“…For decades, certain professions, including surgery, have been associated predominantly with specific genders or backgrounds. 17,18 Age bias can skew representation of certain age groups, whilst body silhouette bias can perpetuate detrimental beauty standards. In this study, Midjourney mainly demonstrated surgeons as older individuals with narrower body silhouettes.…”

Critical review of self‐diagnosis of mental health conditions using artificial intelligence

“…It is crucial that the model can extend its predictions to unobserved data (Murdoch et al, 2019). The generalizability of the model can be determined by evaluating its performance on diverse datasets comprising diverse populations, cultural backgrounds, and age groups (Pagano et al, 2023). A model that performs well across diverse datasets is more likely to be trustworthy and applicable to real-world situations (Acosta et al, 2022;Moor et al, 2023).…”