Measuring object recognition ability: Reliability, validity, and the aggregate z-score approach

Abstract

Measurement of domain-general object recognition ability (o) requires minimization of domainspecific variance. One approach is to model o as a latent variable explaining performance on a battery of tests which differ in task demands and stimuli; however, time and sample requirements may be prohibitive. Alternatively, an aggregate measure of o can be obtained by averaging z-scores across tests. Using data from Sunday et al., Journal of Experimental Psychology: General, 151, 676-694, (2022), we demonstrated that aggregate scores from just two such object recognition tests provide a good approximation (r = .79) of factor scores calculated from a model using a much larger set of tests. Some test combinations produced correlations of up to r = .87 with factor scores. We then revised these tests to reduce testing time, and developed an odd one out task, using a unique object category on nearly every trial, to increase task and stimuli diversity. To validate our measures, 163 participants completed the object recognition tests on two occasions, one month apart.Providing the first evidence that o is stable over time, our short aggregate o measure demonstrated good test-retest reliability (r = .77). The stability of o could not be completely accounted for by intelligence, perceptual speed, and early visual ability. Structural equation modeling suggested that our tests load significantly onto the same latent variable, and revealed that as a latent variable, o is highly stable (r = .93). Aggregation is an efficient method for estimating o, allowing investigation of individual differences in object recognition ability to be more accessible in future studies.

A general ability for judging simple and complex ensembles.

“…The largest correlation was found between the two PS measures. The two OR tests were also correlated as expected from previous studies (Richler et al, 2019;Smithson et al, 2024;Sunday et al, 2018). All EP tests, except for lightness, correlated with all PS and OR tests.…”

“…OR with familiar categories like cars and planes can correlate with age (McGugin et al, 2012; Sunday et al, 2019). Age does not predict OR ability measured with multiple tests and novel objects (Smithson et al, 2024). There are gender differences for OR with certain categories, with an advantage for women for natural categories (e.g., butterflies and leaves) and an advantage for men with manmade categories (e.g., cars and planes, McGugin et al, 2012).…”

“…Two tests were used to measure OR. Following prior work (Chang & Gauthier, 2022; Gauthier & Fiestan, 2023), the tests use novel objects and different task demands, and their aggregate provides an estimate of the latent factor o , as measured with CFA with a larger set of measures (Smithson et al, 2024). One measure was the novel object memory test—greeble (Richler et al, 2017; Figure 5A).…”

“…Based on the findings in a recent study (Chang et al, 2024) that two EP tasks not using the same stimuli, summary statistics, or task format could tap into a common EP ability, we created a series of tests to measure performance in different ensemble judgments for low- and high-level features of various kinds of simple and complex objects. Because many studies have revealed correlations between EP and OR (Cha et al, 2022; Chang et al, 2024; Haberman, Brady, & Alvarez, 2015; Sweeny et al, 2015), we measured domain-general OR, the ability to make within-category discriminations for objects regardless of familiarity and reliance on short- or long-term memory (Richler et al, 2019; Smithson et al, 2024; Sunday et al, 2022). We also chose to control for perceptual speed (PS; Ekstrom et al, 1976).…”

“…PS is a construct related to general intelligence (Ackerman et al, 2002; Redick et al, 2013; Salthouse & Babcock, 1991) and is well-positioned as a potential source of shared variance between EP and OR. In prior studies (Richler et al, 2017, 2019), measures of general intelligence like Raven’s progressive matrices (Raven, 2000) have either not correlated strongly with general OR or were not able to account for all of the stable variance in its measurement (Smithson et al, 2024).…”