Assessment of the equivalence of conventional versus computer administration of the Test of Workplace Essential Skills

Abstract

This study examined the equivalency of computer and conventional versions of the Test of Workplace Essential Skills (TOWES), a test of adult literacy skills in Reading Text, Document Use and Numeracy. Seventy-three college students completed the computer version, and their scores were compared with those who had taken the test in the conventional paper-and-pencil mode. Scores for the two groups for all three subscales were equivalent based on their means and variances. Rank order equivalency was demonstrated for two of the three TOWES subscales (Reading Text and Document Use). Additionally, participants rated the computer version of the test as easy to use.Demonstration of test equivalence is a prevalent issue as paper-and-pencil tests are now frequently adapted for computer administration. Test equivalence means that a person taking the test would get the same score regardless of the test format. Changing formats from paper-and-pencil (conventional) to computer-based tests has been accomplished numerous times with varying degrees of success. The purpose of this study was to assess the degree of equivalence in traditional versus computer administration of the Test of Workplace Essential Skills (TOWES). TOWES assesses literacy skills in the Canadian workplace in three primary areas. Reading Text items assess skill in understanding and using information found in general textual formats includ-

Establishing the Measurement Equivalence of Online Selection Assessments Delivered on Mobile versus Nonmobile Devices

“…First, the support of complete factorial invariance across devices for the learning assessment implies that the factor structure and parameter values were consistent across both medium deliveries for a cognitive ability‐type assessment. This is an important expansion from previous studies that have either not established ME/I for mobile and nonmobile deliveries of cognitive ability assessments (e.g., Chuah, Drasgow, & Roberts, ; Whiting & Kline, ; Schroeders & Wilhelm, ) or have not examined the ME/I for handheld mobile devices in a selection context (e.g., Schroeders & Wilhelm, ). However, before these results can be generalized, a strong word of caution is warranted based on conflicting mean differences evidence gathered in preliminary studies (Doverspike et al, ; Impelman, ).…”

“…As no published research currently exists, it is unclear whether mobile and nonmobile delivered cognitive assessments are equivalent. Comparison studies of computerized versions and traditional paper‐and‐pencil tests have found that technological advancements in computer hardware have eliminated differences that were once caused by functional or contextual characteristics (e.g., legibility and functionality; proctored vs. unproctored; Leeson, ; Noyes & Garland, ; Scott & Mead, ; Whiting & Kline, ). Other research suggests that where these functional or contextual differences are discoverable, they do not have a meaningful impact on performance or equivalence (e.g., Waters & Pommerich, ).…”