The Development of the Attitude towards Computerized Assessment Scale

Abstract

Test equivalence can be evaluated in terms of four aspects: psychometric, behavioral, experiential, and individual differences (i.e., relativity of equivalence) (Honaker, 1988). This study examined the psychometric properties of the Attitude Towards Computerized Assessment Scale (ATCAS) designed to assess one of these criteria, namely, experiential equivalence. Specifically, the ATCAS was developed to explore examinees' emotional, perceptual and attitudinal reactions towards computerized relative to conventional testing methods. The validity and reliability of the ATCAS is addressed by presenting research findings from two independent studies. An exploratory factor analysis (EFA) of the intercorrelations of responses on the 13-items comprising the ATCAS was conducted in Study 1 using data from 92 undergraduate psychology students. The results of the EFA yielded two qualitatively different aspects of the individual's perceptions and experience of computer-based testing (CBT): Ease of Use and CBT-Confidence. The second study used data collected from an Australian Defence Force (ADF) sample to further explore and then confirm the factor structure of the ATCAS. The second study was divided into two phases. Based on the responses of 201 ADF applicants, an EFA of the ATCAS supported the presence of the Ease of Use and CBT-Confidence factors. Based on the responses of 331 ADF applicants, a confirmatory factor analysis (CFA) of ATCAS items supported the two-factor model. In studies 1 and 2, internal consistency was evidenced for the ATCAS and its subscales with values ranging from .76 to .92. Implications for the study of experiential equivalence are discussed, while areas for future research are also considered.

Incorporating computer-assisted assessment to Bachelor of Science in industrial technology courses

“…Early studies showed that the takers without prior computer experience are those likely to have low scores (Bugbee, 1996;Lee, Moreno & Sympson, 1986). This is in contrary to the later findings of other authors stating that the performance between computerized and paper-and-pencil test has no relationship with computer inexperience or anxiety (Mills, 2002;Smith & Caputi, 2004;Wise & Plake, 1989 Draw object is associated with the drawing of simple objects such as lines, and circles. The effectiveness of this method presents very little evidence in the literature most likely due to the fact that common commercial CAA software does not offer them.…”

Past Research in Instructional Technology: Results of a Content Analysis of Empirical Studies Published in Three Prominent Instructional Technology Journals From the Year 2000 Through 2004

“…Quasi-experimental Shapiro & McNamara (2000) To examine the use of latent semantic analysis as a tool for the quantitative assessment of understanding and knowledge Smith & Caputi (2004) To examine the psychometric properties of an assessment scale, which is designed to access the attitudes toward computer based testing Liu et al (2002) To examine the effectiveness of a network method to analyze students' online behavior Rahim, Seyal, & Rahman (2001) To examine the factors affecting softlifting intention of computing students. Dryburg 2000To examine the underrepresentation of girls and women in computer science?…”

Scales assessing L2 speaking anxiety: Development, validation, and application

“…The scales of assessing L2 speaking anxiety, in this case, could be used in tandem with questionnaires evaluating social anxiety or technology anxiety to achieve a well-rounded understanding of all the stressors. Technology-related/testing related scales include the Attitude Toward Computerized Testing Scale (ATCAS) developed by Smith and Caputi ( 2004 ), in which respondents' cognitive and affective reactions toward computerized tests were also assessed.…”