Fairness in Algorithmic Profiling: A German Case Study

Abstract

Algorithmic profiling is increasingly used in the public sector as a means to allocate limited public resources effectively and objectively. One example is the prediction-based statistical profiling of job seekers to guide the allocation of support measures by public employment services. However, empirical evaluations of potential side-effects such as unintended discrimination and fairness concerns are rare. In this study, we compare and evaluate statistical models for predicting job seekers' risk of becoming long-term unemployed with respect to prediction performance, fairness metrics, and vulnerabilities to data analysis decisions. Focusing on Germany as a use case, we evaluate profiling models under realistic conditions by utilizing administrative data on job seekers' employment histories that are routinely collected by German public employment services. Besides showing that these data can be used to predict long-term unemployment with competitive levels of accuracy, we highlight that different classification policies have very different fairness implications. We therefore call for rigorous auditing processes before such models are put to practice.

Structuring the scattered literature on algorithmic profiling in the case of unemployment through a systematic literature review

“…Currently, the dominant data collection methods are administrative data and questionnaires (Desiere et al. , 2019, p. 12), although advances in machine learning increase the availability of big data (Kern et al. , 2021, p. 2; de Troya et al.…”

“…7–14) used the Aequitas toolkit to audit bias and discrimination, finding that the particularly developed algorithm “fails the bias audit”, which suggests researchers “should aim to mitigate bias before implementing” real-world profiling models (p. 14). Similarly, Kern et al. (2021) called for “rigorous auditing processes” before implementation, highlighting how “different classification policies have very different fairness implications” (p. 1).…”

“…Boskoski et al, 2021;de Troya et al, 2018). Finally, the articles comparing the performance of profiling models vary among the disciplines of economics (Lechner and Smith, 2007;Staghøj et al, 2010), computer science (Kern et al, 2021;Zhao, 2020) and public administration (Peck and Scott, 2005).…”

“…7-14) used the Aequitas toolkit to audit bias and discrimination, finding that the particularly developed algorithm "fails the bias audit", which suggests researchers "should aim to mitigate bias before implementing" real-world profiling models (p. 14). Similarly, Kern et al (2021) called for "rigorous auditing processes" before implementation, highlighting how "different classification policies have very different fairness implications" (p. 1). Moreover, "determining the classification threshold cannot be done without consulting a country's socio-institutional context" and policy and statistical experts should accordingly cooperate (Kern et al, 2021, p. 23).…”

“…Table 4). Moreover, Kern et al (2021) stated, "To implement a statistical profiling approach, in reality, one needs to consider many design and policy decisions beyond pure technical details of the statistical prediction model" (p. 23). On the other hand, the examination suggests that literature that politicises the use of profiling and describes the workings of profiling models needs to give more attention to the technical aspects of algorithmic profiling (i.e.…”