Setting Doesn’t Matter Much

Abstract

Abstract. This study deals with the effects of the diagnostic setting on the performance in intelligence tests. We conducted a meta-analysis in which k = 30 samples with a total sample size of N = 2,448 were integrated. We compared results for the same intelligence tests administered either in a group or in an individual setting. The main analysis indicated a small mean population effect [ M( g) = 0.085] that was not significant [−0.036 ≤ M( g) ≤ 0.206]. Nevertheless, moderator analyses indicated a stronger [ M( g) = 0.193] and significant [0.087 ≤ M( g) ≤ 0.298] effect in favor of individual settings for studies employing a between-person design. Setting effects in within-person designs were most likely superimposed by retest effects. As the setting effect was very small, the current testing practice in which results obtained in group and individual settings are treated as interchangeable is not overly problematic. However, our results encourage test developers to examine setting effects before stating that results obtained in different settings are equivalent. Between-person designs using participants of comparable ability are most suitable in this context as retest effects can be ruled out.

Remote Testing of Reading Comprehension in 8-Year-Old Children: Mode and Setting Effects

“…Thus, it could be the case that the individual setting might have played some role because, particularly for complex tasks, the presence of others might impair performance (i.e., the social facilitation phenomenon; Zajonc, 1965). In support of this assumption, a meta-analytic review of studies contrasting individual versus group administrations of intelligence tests showed slightly larger task performance when no other test-takers were present (Becker et al, 2017). On a positive note, the present study found no evidence for device effects in the remote condition.…”

The right amount of pressure: Implementing time pressure in online exams

“…Students writing exams in the comfort of their own home have multiple advantages. Generally, students might be more relaxed in their familiar surroundings, which may lead to increased performance (Becker et al, 2019;Schult et al, 2017), and students with disabilities may experience fewer obstacles to full participation (Kotera et al, 2019;Shevlin et al, 2004). However, studies have also demonstrated that online examinations may increase students' dishonest behavior (Okada et al, 2019).…”

Bereit für Bologna? Test- und notenbasierte Erfolgsprognose im Masterstudium (preprint)

“…Im Gegensatz zu zentralen Zulassungstestungen wurden die Studierenden einzeln von geschulten Interviewern getestet. Die unterschiedliche Situation (Einzel-statt Gruppentestung) sollte jedoch nebensächlich sein, da sich zumindest bei Intelligenztests kein Setting-Unterschied zeigt (Becker, Koch, Schult & Spinath, 2017).…”