Student Accounts of the Ontario Secondary School Literacy Test: A Case for Validation

Abstract

The Ontario Secondary School Literacy Test (OSSLT) is a cross-curricular literacy test issued to all secondary school students in the province of Ontario. The test consists of a reading and a writing component, both of which must be successfully completed for secondary school graduation in Ontario. This study elicited 16 first language and second language student accounts of their OSSLT test-taking processes immediately after the March 2006 test administration. The analysis of these students' accounts provided valuable information about the validity of the inferences drawn from the Ontario Secondary School Literacy Test. These accounts suggest the complexity of the processes that the students engaged as they attempted to demonstrate their reading and writing skills on the test. The study has implications for test developers and test users regarding the interpretation of student test performance on the Ontario Secondary School Literacy Test.Keywords: validation; student accounts; large-scale testing; reading and writing; second language students Résumé : Le Test provincial de compétences linguistiques (TPCL) est un test de littératie transdisciplinaire soumis à tous les élèves des écoles secondaires de l'Ontario. Le TPCL comporte une épreuve de lecture et une épreuve d'écriture qui doivent être réussies pour obtenir le diplô me d'étude secondaire de l'Ontario. L'étude a permis de recueillir des comptes rendus auprès de 16 élèves de langue maternelle (L1) et de langue seconde (L2) sur les processus utilisés pendant le test, immédiatement après la passation de mars 2006. L'analyse de ces comptes rendus a fourni une information utile quant à la validité des inférences à partir du TPCL. Les comptes rendus traduisent la complexité des processus mis en oeuvre alors que les élèves tentent de démontrer leurs habiletés en lecture et en écriture à travers le test. L'étude a des implications pour ceux qui développent le test et les utilisateurs du test en ce qui a trait à l'interprétation de la performance des élèves au TPCL.

Test Takers’ Beliefs and Experiences of a High-stakes Computer-based English Listening and Speaking Test

“…However, these skills were unintended by the test designers. Such findings demonstrate the differences in the views on what a test is testing between test takers and test designers (Cheng et al, 2007;Wijgh, 1996).…”

“…Psychometric approaches are often used to obtain evidence for test validation (Cheng et al, 2007), and such validation evidence appears to come mainly from the test development community (Bachman, 2007). However, what test designers think they are measuring may be different from what test takers think the test is testing (Cheng et al, 2007;Wijgh, 1996). Test takers may correctly answer a question through making a correct hypothesis, a wrong one, or by wild guessing.…”

“…Thus, understanding the test takers' test taking process and strategies via verbal reports can have implications for refining a test instrument and making it more valid (Storey, 1997). Cheng, Fox and Zheng (2007) elicited the accounts of 16 native and non-native English learners concerning their experiences of taking the Ontario Secondary School Literacy Test. These participants provided valuable perceptions on the test content, task properties and the process of taking the test.…”

Access to Creativity: Position of Technology in the Ontario Curriculum for English Language Learners

“…For the OSSLT, success rates are low and deferral rates are high among ELLs (Cheng et al, 2007). For example, based on the 2003 results, 42% of ELLs passed the OSSLT, as compared to the average population success rate of 77% (Cheng et al, 2007).…”

“…For example, based on the 2003 results, 42% of ELLs passed the OSSLT, as compared to the average population success rate of 77% (Cheng et al, 2007). Lotherington (2004) perceived the OSSLT as politically selected literacy.…”