The Validity of Inferences From Locally Developed Assessments Administered Globally

Abstract

In this paper, we first examine the challenges of score comparability associated with the use of assessments that are exported. By exported assessments, we mean assessments that are developed for domestic use and are then administered in other countries in either the same or a different language. Second, we provide suggestions to better support their valid and fair use. We illustrate these issues in the context of higher education assessments that are designed to serve different purposes—inform admissions decisions and assess student learning outcomes within one country (e.g., the United States) that are later used in other countries. In higher education, the use of exported assessments is on the rise due to increases in globalization, student mobility, and cross‐national comparisons of student achievement. An increase in the use of exported assessments leads to more diverse test‐taker populations and requires special attention due to possible sources of construct‐irrelevant variance, which may threaten the score‐based inferences made for various populations. Irrelevant sources of variance may emerge due to differences in opportunity to learn, curricular exposure, and lack of familiarity with the cultural references used in the assessments that are exported.

Linguistic, Contextual, and Experiential Equivalence Issues in the Adaptation of a Performance-Based Assessment of Generic Skills in Higher Education

“…The research literature presents a large number of guidelines and suggestions for adapting test instruments for use in another culture and for evaluating the quality of this process (see Beaton et al, 2000Beaton et al, , 2007Hambleton, 2005;Gudmundsson, 2009;Borsa et al, 2012;Ercikan and Lyons-Thomas, 2013;Hambleton and Lee, 2013;International Test Commission, 2018;Oliveri and Lawless, 2018;Hernández et al, 2020). Nonetheless, researchers have noted that there is no clear agreement on the ideal adaptation method (Borsa et al, 2012;Epstein et al, 2015).…”