Paper–based versus computer–based assessment: key factors associated with the test mode effect

Abstract

This investigation seeks to confirm several key factors in computer-based versus paper-based assessment. Based on earlier research, the factors considered here include content familiarity, computer familiarity, competitiveness, and gender. Following classroom instruction, freshman business undergraduates (N = 105) were randomly assigned to either a computer-based or identical paper-based test. ANOVA of test data showed that the computer-based test group outperformed the paper-based test group. Gender, competitiveness, and computer familiarity were NOT related to this performance difference, though content familiarity was. Higher-attaining students benefited most from computer-based assessment relative to higher-attaining students under paper-based testing. With the current increase in computer-based assessment, instructors and institutions must be aware of and plan for possible test mode effects.As pointed out by other authors in this special issue, for many reasons, the use of computer-based assessment is increasing. Examples include state drivers' license exams, military training exams, job application exams in the private sector, entrance exams in postsecondary education, and certification exams by professional groups (Russo, 2002;Trotter, 2001). However, in the literature, there is mounting empirical evidence that identical paper-based and computer-based tests will not obtain the same results. Such findings are referred to as the "test mode effect."For example, paper-based test scores were greater than computer-based test scores for both mathematics and English CLEP tests (Mazzeo, Druesne, Raffeld, Checketts and Muhlstein, 1991) and for recognizing fighter plane silhouettes (Federico, 1989); while computer-based test scores were greater than paper-based test scores for a dental hygiene course unit midterm examination (DeAngelis, 2000); though other studies

Using mobile and web‐based computerized tests to evaluate university students

“…Computer-based testing (CBT) that simply uses personal computers (PCs) as the medium for students to take exams, for teachers to prepare tests, and for the transmission of traditional paper and pencil tests (PPTs) [3]. Compared to PPTs, CBTs offer logistical advantages, a unified platform for media, diverse item types, the tracing of test-taker actions, and immediate feedback.…”

Analysing Test-Takers’ Views on a Computer-Based Speaking Test

“…The technological simplicity of the aptis test may seem to be an advantage for most test-takers (see Kenyon & Malabonga, 2001) who appeared to feel comfortable with the management of the new software. This is an important aspect to be taken into account since various research findings suggest that computer familiarity and other features of the computerised context (e.g., computer anxiety) may affect candidates' performance (Chapelle, 2001;Clariana & Wallace, 2002;Colwell, 2013;J. A. Lee, 1986;Norris, 2001;Taylor, Kirsch, Jamieson, & Eignor, 1999).…”

Dual modes for written assessments: Examining validity and reliability

“…Clariana and Wallace (2002) compared computer-based and paper-based test scores for a multiplechoice test for first-year tertiary students in a business programme in the USA. The computer-based group (54 students) scored a mean of 83 percent (a B overall), while the paper-based group (51 students) scored a mean of 76 percent (a C overall).…”