Assessing the Validity of Computer-Based Test Interpretations: Rating Reliability and Individual Differences Among Raters

Guastello, Stephen J.; Rieke, Mark L.; Guastello, Denise D.; Billings, Steven W.

doi:10.1207/s15327752jpa5801_7

Cited by 3 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Guastello (Endres, Guastello and Rieke 1992;Guastello and Rieke 1990;Guastello, Rieke, Guastello and Billings 1992;Prince and Guastello 1990) has discussed a number of methodological issues associated with attempts to measure the Barnum effect in relation to CBTI reports and presented a number of evaluations showing positive differences between genuine and bogus reports for tests like the 16PF. However, Standing and Keays (1986), who examined reports generated by the program Mind Prober, found that spurious reports were rated as highly for accuracy as the genuine descriptions.…”

Section: Users and Usesmentioning

confidence: 99%

The Role of Computer‐Based Test Interpretation (CBTI) in Occupational Assessment

Bartram

1995

Int J Selection Assessment

View full text Add to dashboard Cite

The paper explores a range of issues relating to the design and use of computer-based test interpretation (CBTI) systems. It describes a number of dimensions along which CBTI systems vary and considers both the positive and negative implications of these for test users, for their clients and for test candidates. A distinction is drawn between systems that generate finished reports and those that generate questions or guidance for the user. The question of how the validity of narrative reports can be assessed is considered. A study on the discriminative validity of reports generated by the PREVUE ICES CBTI system is reported which shows clear differences between judgements made about true and bogus reports. It is concluded that there are two main issues to consider when evaluating a CBTI system: first, the validity of the information generated by it and, second, the ways in which that information is likely to be used. ver the past decade, three main issues have emerged in the computer-based The use of CBTI has grown enormously since Fowler (1972) commented on the lack of Personality test CBTI systems constitute by far assessment (CBA) literature as being key substantial progress made in the field. concerns of practitioners: 0 (1) The equivalence of the same test presented in paper-and-pencil format and on computer. (2) The development and implementation of adaptive testing. (3) The use of computers to generate interpretive reports -especially of personality inventories.The present paper limits its focus to the third of these issues: computer-based test interpretation (CBTI). It is here that computers have had their greatest impact on assessment practice in occupational psychology. Issues of test equivalence, whilst technically important, are not controversial, and adaptive testing has yet to make an impact on the day-to-day work of more than a handful of assessment practitioners. The present author has provided a number of reviews of CBA (Bartram 1987, 1993a, 1994Bartram and Bayliss 1984) from an occupational psychology viewpoint which have gone beyond consideration of these three issues and explored some exciting possibilities for assessment opened up by new technology (other recent wide-ranging reviews include that by Jager (1991) and a useful bibliography of recent research is presented by Argentero (1989, (in Italian)). However, novel forms of computer testing have also had little impact on most practitioners, and are still largely confined to specialist military use (e.g. Burke and Van Raay 1993).

show abstract