A Conceptual Framework for Investigating and Mitigating Machine-Learning Measurement Bias (MLMB) in Psychological Assessment

Abstract

Given significant concerns about fairness and bias in the use of artificial intelligence (AI) and machine learning (ML) for psychological assessment, we provide a conceptual framework for investigating and mitigating machine-learning measurement bias (MLMB) from a psychometric perspective. MLMB is defined as differential functioning of the trained ML model between subgroups. MLMB manifests empirically when a trained ML model produces different predicted score levels for different subgroups (e.g., race, gender) despite them having the same ground-truth levels for the underlying construct of interest (e.g., personality) and/or when the model yields differential predictive accuracies across the subgroups. Because the development of ML models involves both data and algorithms, both biased data and algorithm-training bias are potential sources of MLMB. Data bias can occur in the form of nonequivalence between subgroups in the ground truth, platform-based construct, behavioral expression, and/or feature computing. Algorithm-training bias can occur when algorithms are developed with nonequivalence in the relation between extracted features and ground truth (i.e., algorithm features are differentially used, weighted, or transformed between subgroups). We explain how these potential sources of bias may manifest during ML model development and share initial ideas for mitigating them, including recognizing that new statistical and algorithmic procedures need to be developed. We also discuss how this framework clarifies MLMB but does not reduce the complexity of the issue.

Toward Argument‐Based Fairness with an Application to AI‐Enhanced Educational Assessments

“…Our proposed argument-based fairness approach mirrors the process of argumentbased validity in many ways as it follows parts of the Toulmin model of argument structure (1958/2003). To introduce the approach before couching it within AI assessment, we use some isolated examples from ongoing work developing a classroom-based reading assessment in the Institute of Education Science's funded Project DIMES (Huggins-Manley, Benedict, Goodwin, & Templin, 2019-2022.…”

“…This work established methods and metrics for quantifying and studying bias, aligning such methods to demarked stages of such assessment development. More broadly, Tay et al (2022) proposed a framework for conceptualizing and quantifying bias in AI assessments that use machine-learning (ML) as the core assessment engine, discussing some distinctions between fairness and bias and then focusing the framework on the latter. They, too, centered their approach to fairness on matters of bias only and aligned the bias concerns to stages of assessment development, which are somewhat different than traditional assessments as elaborated later and discussed in D' Mello et al (2021).…”

“…Whereas the above proposed approach to argument-based fairness holds for both traditional and AI assessments, it is the stages of assessment, types of claims, and types of evidence that can separate the latter from the former, at a minimum. As such, it is important to first identify the unique features of AI assessments as discussed extensively elsewhere (D'Mello et al, 2021;Tay et al, 2022) and summarized below:…”

“…Typically, this is done using a form of machine learning (ML) called supervised learning where the ML algorithm uses a human-provided supervisory signal to learn the mappings from low-level signals into higher-level abstractions and then to the construct(s) (e.g., identifying a face in an image; identifying a furrowed brow in a face; learning that a furrowed brow can indicate confusion). Due to the heavy reliance on ML, these assessments should be termed ML-enhanced assessments as in Tay et al (2022), but we use the broader term AI assessments here.…”

“…Based on current approaches to fairness in AI assessment (Booth et al, 2021a;Tay et al, 2022) and based on many guidelines for fair assessment (e.g., Standards [APA et al, 2014], Code of Fair Testing Practices in Education (Joint Committee on Testing Practices, 2004), Standards for Fairness and Quality (Educational Testing Service, 2002), we recommend that warrants in fairness arguments be couched within the stages of assessment. We adapt Tay et al's (2022) analysis when considering the stages shown in Figure 2, aligned with the construct, task, and evidence models espoused by evidence-centered design (ECD) (Mislevy & Haertel, 2006). The stages include:…”