Comparability of a paper-based language test and a computer-based language test

Abstract

With the advent of the digital revolution, language testers have endeavored to utilize state-of-the-art computer technology to satisfy the ever-growing need for a tool to measure English communication skills with maximal accuracy and efficiency. Thanks to the concerted efforts made by experts in such fields as computational linguistics, computer engineering, computer-assisted language learning, and psychometrics, language testers have recently succeeded in developing computer/web-based language tests. Among them are the TOEFL CBT by Educational Testing Service and CommuniCAT by the University of Cambridge Local Examinations Syndicate. As with the paper-based language test (PBLT), more rigorous research is now being conducted on the validity of computer-based language tests (CBLT) and computer adaptive language tests (CALT). Content analyses and comparability studies of PBLT and CBLT/CALT are prerequisites to such validation research. In this context, utilizing an EFL test battery entitled the Test of English Proficiency developed by Seoul National University (TEPS), the present study is aimed at addressing the issue of the comparability between PBLT and CBLT based on content and construct validation employing content analyses based on corpus linguistic techniques in addition to such statistical analyses as correlational analyses, ANOVA, and confirmatory factor analyses. The findings support comparability between the CBLT version and the PBLT version of the TEPS subtests (listening comprehension, grammar, vocabulary, and reading comprehension) in question.

Cross-Mode Comparability of Computer-Based Testing (CBT) Versus Paper-Pencil Based Testing (PPT): An Investigation of Testing Administration Mode among Iranian Intermediate EFL Learners

“…Furthermore, Boo et al (2012) found that although test takers preferred computer counterpart of the conventional test, the scores received from CBT and PPT were comparable in terms of internal consistency, criterion and construct validities, means and standard deviations. Choi, Kim, and Boo (2003) reported that the results of paper and computer versions of the standardized English language test administered to post-secondary level language learners were comparable. They proved that two versions of listening, reading comprehension, grammar and vocabulary subtests measured the same constructs based on confirmatory factor analysis results.…”

Doubts on the validity of correlation as a validation tool in second language testing research: the case of cloze testing

“…There are numerous other studies in which the technique of correlation has been used correctly. The most recent studies in this category are (Choi et al 2003), (Trites and McGroatry 2005) and (Roever 2006), to name a few. (Roever 2006), for instance, compared web-based tests of implicatures, routines and speech-acts and found that the tests were related, without claiming that they could be substituted.…”

Comparative Study on Validity of Paper-Based Test and Computer-Based Test in the Context of Educational and Psychological Assessment among Arab Students

“…The related notion of correspondent results both is PBT and CBT was also reinforced by many studies in certain specific subject areas, and the clear discrimination of results was established in achievement tests such as science, language and mathematics, and also the same was very perceptibly ascertained by a chain of psychological tests such as personality and neuropsychological assessment (e.g., Friedrich & Bjornsson, 2008;Choi, Kim, & Boo, 2003;DeAngelis, 2000). In their findings about the review of educational and psychological measurement approaches, Bunderson, Inouye & Olsen (1989) have established that 48% of previous studies revealed negligible difference between the two testing modes (PBT & CBT) in the area of test performance, whereas 13% of studies have reported that CBT test performance is better than PBT and 39% of findings have proved PBT better than CBT.…”