Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

Abstract

The increasing use of data-driven decision support systems in industry and governments is accompanied by the discovery of a plethora of bias and unfairness issues in the outputs of these systems. Multiple computer science communities, and especially machine learning, have started to tackle this problem, often developing algorithmic solutions to mitigate biases to obtain fairer outputs. However, one of the core underlying causes for unfairness is bias in training data which is not fully covered by such approaches. Especially, bias in data is not yet a central topic in data engineering and management research. We survey research on bias and unfairness in several computer science domains, distinguishing between data management publications and other domains. This covers the creation of fairness metrics, fairness identification, and mitigation methods, software engineering approaches and biases in crowdsourcing activities. We identify relevant research gaps and show which data management activities could be repurposed to handle biases and which ones might reinforce such biases. In the second part, we argue for a novel data-centered approach overcoming the limitations of current algorithmic-centered methods. This approach focuses on eliciting and enforcing fairness requirements and constraints on data that systems are trained, validated, and used on. We argue for the need to extend database management systems to handle such constraints and mitigation methods. We discuss the associated future research directions regarding algorithms, formalization, modelling, users, and systems.

[Retracted] Applications of Deep Learning in the Evaluation and Analysis of College Students’ Mental Health

“…At present, in the research process of mental health assessment and analysis of a large number of college students in China, due to the current situation of different regional cultures and different family economic levels, there will be problems of analysis difficulty and inaccurate analysis [5]. It is difficult for scholars at home and abroad to deal with the group of college students in the research of psychology, because this stage of college students is a stage of rapid change in life and prominent life process, so the research situation is more one-sided and concentrated, and there is no general, comprehensive, universal, and all-round research on college students' mental health [6]. Biswas et al found that the current mental health status of college students is closely related to their environment, their own pressure, man-machine communication relationship, family economic level, and other factors.…”

Decolonising global health by decolonising academic publishing

“…As the world increasingly becomes data-driven and machine-learning evaluated, it is essential to anticipate issues of bias and fairness, and proactively devise ways to counter existing and entrenched biases in data collection methods 20 21. Logging transparent data, in deliberately designed data structures, is likely to help address harmful and ethically dubious recommendations and conclusions based on biased data 21 22…”

A fuzzy-rough uncertainty measure to discover bias encoded explicitly or implicitly in features of structured pattern classification datasets

“…Data-driven decision support systems have been accused of being a fertile ground to produce biased results, thus leading to discriminatory decisions [1]. As historical data often encode biases [2] explicitly or implicitly [3], pattern recognition algorithms inevitably relate their predictions with protected characteristics such as race or gender.…”

“…In the literature, more than 20 definitions of fairness [4] and respective bias metrics have been proposed. However, existing metrics express different and often contradictory notions of fairness [5,6,7] depending on local legal and cultural conventions [8] or on the type of decisionsupport system [1]. Deciding which metric is most appropriate for the task at hand is difficult [9] as several parameters need to be considered such as causal influences among features, misrepresentation of groups and different modalities of data [4].…”

“…1 Pearson correlation between protected and unprotected features. Asterisks indicate significant p-value (p < 0.05) or F-statistic (larger than the critical value) 2.…”