Comparability of Computer and Paper‐and‐pencil Scores for Two Clep® General Examinations

Abstract

This report describes two studies that investigated the comparability of scores from paper‐and‐pencil and computer‐administered versions of the College‐Level Examination Program (CLEP) General Examinations in Mathematics and English Composition. The first study used a prototype computer‐administered version of each examination. Based on the results of the first study and feedback from the study participants, several modifications were made to these prototype versions. A second study was then conducted using the modified computer versions. Both studies used a single‐group counterbalanced equating design. Data for the Mathematics Examination were collected at Southwest Texas State University, and data for the English Composition Examination were collected at Utah State University. The results of Study 1 suggest that, despite efforts to design computer versions of the CLEP Mathematics and English Composition General Examinations that were administratively similar to the paper‐and‐pencil examinations (i.e., allowed item review and answer changing and were comparably timed), mode‐of‐administration effects (i.e., changes in average scores as a function of the mode of test delivery) were found. The results of Study 2 suggest that the modifications made to the computer versions eliminated the mode‐of‐administration effects for the English Composition Examination but not for the Mathematics Examination. The results of both studies underscore the need to determine empirically (rather than to just assume) the equivalence of computer and paper versions of an examination.

Computer-based Testing and Validity: a look back into the future

“…In a later study, Mazzeo et al (1991) reaffirmed the need to determine empirically the equivalence of computer and paper versions of an examination. Additionally, Kolen and Brennan (1995) argue that mode effects of paper-and-pencil and computer-based tests are complex and that the extent to which effects are present are likely dependent on the particular testing programme.…”

“…This complexity also suggests that separate analyses for modal sensitivity are necessary for any test offered in both formats. Finally, Mazzeo et al (1991) recommended examining relative effects of mode of administration among subpopulation groups, which, over a decade later, is still not a factor commonly studied.…”

Comparing Online and Paper Exams: Performances and Perceptions of Saudi Students

“…They measured students' perceptions toward the use of an online exam as an assessment tool on university campus within a Decision Support System Course at Al al Bayt University [7]. Mazzeo, Druesne, Raffeld, Checketts, & Muhlstein (1991) showed that paper-based test scores were greater than computer-based test scores for both mathematics and English CLEP tests [8]; although studies of Schaeffer, Reese, Steffen, McKinley, & Mills (1993) have reported no difference between computer and paper-based tests [9]. This paper presents the results of comparative study between paper based and online exams conducted on a group of Tabuk university students, who had no prior experience of taking online exams, in the region of Tabuk, Saudi Arabia.…”

Development of a Scale for Assessing the Level of Computer Familiarity of Toefl Examinees

“…Although Lee had intended to create an interval-level scale based on these questions, she reported that "due to skewness, restricted variance, and low Cronbach's alpha, it was decided to use the CEQ scores to form a categorical variable" (p. 729). The CEQ was subsequently modified and used in two studies by Mazzeo, Druesne, Raffeld, Checketts, and Muhlstein (1991). Unlike Lee's scale, Mazzeo et al summed the values ofthe responses, creating an interval-level scale with values ranging from 6 to 27; this scale was not analyzed for its psychometric properties.…”