A knowledge-based theory of rising scores on “culture-free” tests.

Abstract

Secular gains in intelligence test scores have perplexed researchers since they were documented by Flynn (1984, 1987). Gains are most pronounced on abstract, so-called culture-free tests, prompting Flynn (2007) to attribute them to problem-solving skills availed by scientifically advanced cultures. We propose that recent-born individuals have adopted an approach to analogy that enables them to infer higher level relations requiring roles that are not intrinsic to the objects that constitute initial representations of items. This proposal is translated into item-specific predictions about differences between cohorts in pass rates and item-response patterns on the Raven's Matrices (Flynn, 1987), a seemingly culture-free test that registers the largest Flynn effect. Consistent with predictions, archival data reveal that individuals born around 1940 are less able to map objects at higher levels of relational abstraction than individuals born around 1990. Polytomous Rasch models verify predicted violations of measurement invariance, as raw scores are found to underestimate the number of analogical rules inferred by members of the earlier cohort relative to members of the later cohort who achieve the same overall score. The work provides a plausible cognitive account of the Flynn effect, furthers understanding of the cognition of matrix reasoning, and underscores the need to consider how test-takers select item responses.

The magical numbers 7 and 4 are resistant to the Flynn effect: No evidence for increases in forward or backward recall across 85 years of data

“…In fact, it is possible that the results of this investigation may be considered in line with the contention that the Flynn effect is operating primarily at the level of abstraction ability (Flynn, 2012;Fox & Mitchum, 2013), rather than on a test such as Digit Span, as Digit Span is based on stimuli to which individuals 85 years ago and today would have about an equal amount of exposure, i.e., digits from one to nine. Such a contention may be considered ostensibly plausible, however, when examined thoroughly, one would draw the conclusion that humans have been exposed to digits at a substantially increasing rate across time.…”

“…Noteworthy, however, is the item-level research which suggests that the Flynn effect associated with Raven's scores may be due principally to cohort differences in the capacity for abstraction (Fox & Mitchum, 2013). Based on the results of this investigation, it would appear that any possible increases in abstraction capacity across time have occurred completely devoid of any increases in WMC.…”

Recent evolution of learnability in American English from 1800 to 2000

“…In addition, language efficiency is likely to have pronounced effects on the cognitive capacities of its users. For example, the documented rise in symbolic intelligence associated with the Flynn effect (Flynn, 2012;Fox & Mitchum, 2013) could be both cause and consequence of increased concreteness in language. That is, an increase in concrete language may have enhanced our individual capacity to process complex information (Sadoski, 2001) at the same time as it allows us to communicate clear but increasingly complicated ideas.…”

The importance of measurement invariance in neurocognitive ability testing

“…Factors causing bias in an entire subtest can be diverse and relate to practical problems in how the test is administered, cultural issues related to item or test content, educational differences, language issues, but also cognitive or emotional factors. For instance, test anxiety due to negative stereotypes might cause bias on mathematical tests (Wicherts, Dolan, & Hessen, 2005), and test-taking strategies can greatly alter how one takes a fluid reasoning task such as the raven's progressive Matrices tests (Fox & Mitchum, 2012). If such strategy use differs across groups, we would expect invariance to fail.…”