On test development and accuracy in self-assessment

Abstract

EDITORIALOn test development and accuracy in self-assessmentThe opening article by Jin and Hamp-Lyons (2015) in this regular issue reports on an empirical investigation of the need for a new test for the assessment of English for professional purposes in China. As the authors point out, it is not often the need for a new test is investigated empirically; tests are usually introduced by educational policy-makers or suggested by commercial organisations for the test's potential market value and profits. With reference to Shohamy (2001), Jin and Hamp-Lyons remind us that the voices of stakeholders such as the learners, teachers and potential users of the test are seldom heard. The article therefore offers an important contribution to the field of language testing by reporting the initial phases of the development of a new language assessment. In the first phase, an online survey was conducted collecting data on stakeholders' perceptions of the necessity of the test (N = 327), representing three stakeholder groups: (1) university students, (2) university graduates employed by joint ventures or multinational companies and (3) university teachers of advanced English courses. In the second phase, the researchers analysed five English language tests, (Business English Certificate -Higher, Test of English for International Communication (TOEIC), CET-6, Test for English Majors and Public English Test System) to see whether existing tests would fit the need for the new test, or inform the development of the new proposed test. The first two tests are developed for an international marked, while the final three are developed for China. Finally, the researchers reviewed six language frameworks (Foreign Service Institute, or Interagency Language Table, Australian Second Language Proficiency Ratings (ASLPR), Proficiency Guidelines of American Council on the Teaching of Foreign Languages, Can-Do statements of Association of Language Testers in Europe, CEFR, and Canadian Language Benchmarks) to evaluate whether any of them could serve as guidelines for the design of the new assessment, particularly the test's specifications. Data analysis of the survey revealed some discrepancies among the stakeholder groups on the perceptions of the needs of communication in English in China, but all groups agreed on the need to develop a test of EPP for Chinese learners of English at an advanced level. The analysis of the five tests further showed that none of the Chinese tests on the marked were sophisticated enough for the purpose of assessing English proficiency at an advanced level for multinational professional workplaces in China. The review of the existing frameworks also showed that the level descriptions were too general for any practical use for the test developers, and more related to vocational purposes than professional purposes. The authors therefore argued for the need to develop a new test in China with real value for potential employers. The study is a timely reminder of the importance for researchers and test-designers to inves...