Problems and Issues in Translating International Educational Achievement Tests

Abstract

The article reviews research and findings on problems and issues faced when translating international academic achievement tests. The purpose is to draw attention to the problems, to help to develop the procedures followed when translating the tests, and to provide suggestions for further research. The problems concentrate on the following: the unique and demanding purpose of the translation task, the partly contradictory task specifications and translation instructions, the indecision as to whether to produce one or two target versions, the indecision as to whether to use one or two source versions, inadequate revision and verification, deficient translator competences, and a lack of time. To solve the problems, the article suggests the following: ensuring that the translation guidelines provide a right, unequivocal, and balanced picture of the purpose of the translation task; ensuring the equivalence of the two source versions; putting more emphasis on revision, and ensuring that the verification is sufficiently thorough; using only qualified translators, providing them with training in test translation, and including also subject matter and testing specialists in the translation teams; and allotting sufficient time to the translation work. However, the main lesson from the review is that more research in the field is badly needed.

The translator’s perspective on translation quality control processes for international large-scale assessment studies

“…The interviews indicated that in iLSA, translators may not receive sufficient information throughout the process to make confident translation decisions. This observation is in line with the findings of Arffman (2013). Translators who are only involved in the translation step (and do not receive extensive training, further feedback or participate in later steps) simply produce text and must rely on the information they receive in the form of general or item-specific guidelines.…”

Measurement Comparability of Reading in the English and French Canadian Populations: Special Case of the 2011 Progress in International Reading Literacy Study

“…The purpose of difficulty equivalence is to minimize differences across language versions related to required cognitive effort. These dual purposes increase the difficulty of the translation task, and translators are usually not trained to pursue difficulty equivalence (Arffman, 2013). Results from this study and those from Arffman's research on translation procedures utilized by the IEA may indicate that it would be prudent to review current practices for adapting PIRLS in Canada to provide information about the strengths and weaknesses of existing practices and indicate how to create systematic approaches to ensure test equity for French and English language groups.…”

“…Although translated versions for each country undergo two rounds of verification reviews by linguistic and assessment experts at the international test center, translation procedures are at the discretion of national test centers. Research demonstrates that guidelines are insufficient to ensure high-quality adaptation (Solano-Flores et al, 2009;Arffman, 2010Arffman, , 2013. Arffman (2013) argues that the process used to translate IEA studies do not necessarily align with the principles and procedures of translation.…”

“…Research demonstrates that guidelines are insufficient to ensure high-quality adaptation (Solano-Flores et al, 2009;Arffman, 2010Arffman, , 2013. Arffman (2013) argues that the process used to translate IEA studies do not necessarily align with the principles and procedures of translation. To address this shortcoming, Arffman recommends that IEA guidelines provide more detailed instructions, with examples to illustrate the delicate tension between two translation purposes that are crucial to international test translation.…”

Examining Cultural Responsiveness in Large-Scale Assessment: The Matrix of Evidence for Validity Argumentation

“…This underrepresentation of other cultures can potentially have an adverse impact on the performance of students from cultures that are not those portrayed in the test items. Some cultural adaptations are made on the characteristics of items when they are translated into the languages used in the participating countries (see Arffman, 2013;OECD, 2017). However, these adaptations tend to be few and superficial and do not include the contextual information of items.…”