Repurposing a Business Learning Outcomes Assessment to College Students Outside of the United States: Validity and Reliability Evidence

Abstract

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.Find other ETS-published reports by searching the ETS ReSEARCHER database at http://search.ets.org/researcher/ To obtain a copy of an ETS research report, please visit AbstractOne concern when repurposing a test to a new population is whether the test is measuring the same construct in a valid and reliable way that is comparable to the intended population. Following the guidelines of the International Test Commission and the ETS Standards forQuality and Fairness, this study was designed to collect evidence in support of repurposing the Major Field Test for the Bachelor's Degree in Business (MFT-B) for use in undergraduate level business programs outside of the United States. The author examined the test's reliability and internal structure based on 930 senior college students from 9 business programs in Asia, Europe, the Middle East, and South America. Each participating institution satisfied 2 requirements for this study: (a) having a bachelor's level business curriculum that is similar to their U.S. peer programs and (b) using English as the instructional language. The analyses of the data showed that the MFT-B has the same factor structure for the non-U.S.-based sample as that for their U.S. peers. In addition, the test scores for the non-U.S.-based sample are highly reliable (.90) overall, with some variations among the 9 programs. Both types of evidence support the notion that the MFT-B is appropriate when repurposed for use with non-U.S.-based business programs.

The Validity of Inferences From Locally Developed Assessments Administered Globally

“…versus U.S.‐based programs when an assessment is given outside the United States. Ling () also demonstrated that regional differences may appear at the program level. The extent to which the program‐level variations of non‐U.S.–based programs resemble those of the U.S. programs remains a question worthy of investigation as one considers exporting assessments.…”

“…Ling () pointed to curricular differences, which may impact performance and interpretations of score differences for non‐U.S. versus U.S.‐based programs when an assessment is given outside the United States.…”

“…They also may be due to possible differences in opportunity to learn because of a different emphasis (inclusion or exclusion) of a particular curricular component in the curriculum of one country as compared to another . Ling (2013) pointed to curricular differences, which may impact performance and interpretations of score differences for non-U.S. versus U.S.-based programs when an assessment is given outside the United States. Ling (2013) also demonstrated that regional differences may appear at the program level.…”

“…Such uses may be acceptable for admissions into a U.S. law school but not for admission into a law school in a different country, whereas other (more general) aspects of the test may be deemed suitable for both countries. Along the same lines, Ling () provided an example with respect to an SLO assessment that tests business skills: “the content covered or focused on in non‐U.S.‐based business programs may slightly differ from that in the United States, mainly because the business context in these countries and regions differs from the U.S. context to varying degrees” (p. 9). In such instances, curricular experts in the new country may want to identify which components are applicable to their country and which ones are less so.…”