Social impacts of algorithmic decision-making: A research agenda for the social sciences

Abstract

Academic and public debates are increasingly concerned with the question whether and how algorithmic decision-making (ADM) may reinforce social inequality. Most previous research on this topic originates from computer science. The social sciences, however, have huge potentials to contribute to research on social consequences of ADM. Based on a process model of ADM systems, we demonstrate how social sciences may advance the literature on the impacts of ADM on social inequality by uncovering and mitigating biases in training data, by understanding data processing and analysis, as well as by studying social contexts of algorithms in practice. Furthermore, we show that fairness notions need to be evaluated with respect to specific outcomes of ADM systems and with respect to concrete social contexts. Social sciences may evaluate how individuals handle algorithmic decisions in practice and how single decisions aggregate to macro social outcomes. In this overview, we highlight how social sciences can apply their knowledge on social stratification and on substantive domains of ADM applications to advance the understanding of social impacts of ADM.

Tweaking algorithms. Technopolitical issues associated with artificial intelligence based tuberculosis detection in global health

“…Two biases are often considered in practice: A so-called over-reliance bias and an algorithmic aversion bias, i.e., a complete refusal to use algorithms. 11 , 12 The process of building trust in the tool is a gradual one, involving many players. The first stage in this process of building trust has been ensured by numerous scientific publications evaluating CAD.…”

Race-neutral vs race-conscious: Using algorithmic methods to evaluate the reparative potential of housing programs

“…Thus, the algorithmic fairness discourse may be limited because in many cases, machine learning algorithms utilize the data at the point of creating the algorithm without considering the historical context in which the input data were generated (So et al, 2022). This can lead to machine learning models that "learn" to reinforce disparities that were created by seemingly race-neutral markers as objective truths, thereby legitimizing different treatments (Benjamin, 2019;Browne, 2010;Gerdon et al, 2022). Examples of this process in the domain of housing include the use of seemingly race-neutral variables, particularly risk-based pricing algorithms, such as security deposits in the rental market (Hatch, 2017), and mortgage insurance in mortgage loans (Deng and Gabriel, 2006).…”

Editorial: Big data and machine learning in sociology

“…Algorithms can also be sensitive to contextually problematic conceptualizations and depend on interactional settings. This can be highly impactful for the generation and reproduction of social inequalities as “one of the core competencies—and responsibilities—of the social sciences” (Gerdon et al, 2022 , p. 2; see also Section 4).…”