Psychometric and Psychological Effects of Item Selection and Review on Computerized Testing

Abstract

Psychometric properties of computerized testing, together with anxiety and comfort of examinees, are investigated in relation to item selection routine and the opportunity for response review.

The instinct fallacy: the metacognition of answering and revising during college exams

“…Students preferred the review condition, but did not benefit from it in terms of performance. However Revuelta et al (2003) did find an improvement using a similar computerized English test; their participants also preferred reviewing to not reviewing. Overall, the results suggest that students are hesitant to change answers -usually only a few on an entire exam will be changed -and thus get only a small improvement from the behavior.…”

“…When past researchers have attempted to determine the reasons behind and benefits of revising, the results were murky at best (Benjamin et al 1984;Crawford 1928;Crocker and Benson 1980;Foote and Belinky 1972;Geiger 1996;Harvil and Davis 1997;Higham and Gerrard 2005;McMorris et al 1987;Mueller and Wasser 1977;Revuelta et al 2003;Shatz and Best 1987;Vispoel 1998Vispoel , 2000Vuk and Morse 2012). Shatz and Best (1987) found that changing an answer because it had originally been a guess -the most cognitively important change -was not nearly as beneficial as changing because of simple reasons such as misreading the question or accidentally marking the wrong choice.…”

Assessing scientific literacy through computer-based tests – consequences related to content and gender

“…Research has shown that low screen resolution has a negative impact on reading tests but not on mathematics tests (Bridgeman, Lennon, & Jackenthal, 2002), that medium line length increases reading comprehension compared to short and long lengths (Dyson & Haselgrove, 2001), and that the possibility to review individual items usually improves the respondent's results (Revuelta, Ximénez, & Olea, 2003). However, Leeson (2006) argues that, although the availability for item review is taken for granted in paper-and-pencil-based tests, it is usually not an option when taking the test on computers and definitely not an option on computer adaptive tests, where the next item on the test is selected on the basis of previous answers.…”

Felső tagozatos diákok papír-ceruza és számítógép alapú teszteredményeinek összehasonlító vizsgálata szövegértés, induktív gondolkodás és problémamegoldás terén

“…Higgins A tesztmegoldás, a teszttel folytatott interakció vizsgálata során az előző feladatokra történő visszalépés lehetőségét (flexibilitást) is vizsgálták a számítógépes tesztelés tekintetében (Bodmann és Robinson, 2004;Revuelta, Ximénez és Olea, 2003;Vispoel, 1998Vispoel, , 2000. A nemzetközi kutatások eredményei alapján, amennyiben a feladatmegoldás során nem kötelező a válaszadás, biztosított a korábbi feladatra történő visszalépés és válasz megváltoztatásának lehetősége, akkor a tesztkörnyezet flexibilitásának nincs jelentős hatása a diákok online tesztpontszámaira (Bodmann és Robinson, 2004;Vispoel, 2000).…”