Human memory for real-world solid objects is not predicted by responses to image displays.

Abstract

In experimental psychology and neuroscience, computerized image stimuli are typically used as artificial proxies for real-world objects to understand brain and behavior. Here, in a series of five experiments (n = 165), we studied human memory for objects presented as tangible solids versus computerized images. We found that recall for solids was superior to images, both immediately after learning, and after a 24-hr delay. A "realness advantage" was also evident relative to three-dimensional (3-D) stereoscopic images, and when solids were viewed monocularly, arguing against explanations based on the presence of binocular depth cues in the stimulus. Critically, memory for solids was modulated by physical distance, with superior recall for objects positioned within versus outside of observers' reach, whereas recall for images was unaffected by distance. We conclude that solids are processed quantitatively and qualitatively differently in episodic memory than are images, suggesting caution in assuming that artifice can always substitute for reality. Public Significance StatementScientists study vision, thinking, and memory by running controlled laboratory experiments; results from the laboratory are then generalized to understand brain function in real-world contexts. In laboratory studies of human memory, scientists measure responses to visual stimuli (such as common objects) which are typically presented as images on a computer screen, rather than as real-world solid objects. Here, we show that memory for real-world objects cannot be predicted based on responses to artificial image displays. This work suggests that stimulus realism influences how information is processed in the brain, questioning how well results from studies of artificial stimuli generalize to explain behavior in the natural world.

Naturalistic Object Representations Depend on Distance and Size Cues

“…Therefore, to explore the extent to which other stimulus features influenced representations over and above egocentric distance, we split the data between the two egocentric distances and conducted RSA and MDS analyses of object representations separately at the near and far stimulus locations. Finally, because accumulating evidence from behavior and neuroimaging studies suggest that stimulus size is processed more readily in the case of real objects than of pictures (Holler et al, 2019;Sensoy et al, 2021), and that stimulus distance modulates responses to real objects (Gallivan et al, 2009(Gallivan et al, , 2011Cavina-Pratesi et al, 2010) but less so for pictures (Gomez et al, 2018;Snow et al, 2023), we further split the data according to display format to examine whether stimulus size influenced representational patterns differently according to the combination of display format and egocentric location. We focus here on ROIs defined in the left hemisphere, consistent with the hemisphere of motoric dominance of our right-handed participants; however, the pattern of results in the right hemisphere (Supplementary Figures S1-5) was strikingly similar to that in the left.…”

“…From a theoretical perspective, real objects presented at a near egocentric position are uniquely relevant for manual interaction, whereas this is less so for real objects presented at a far position (because they are outside of reach), and for pictures at either location (because pictures do not afford manual actions). From an empirical perspective there is increasing evidence that real objects elicit different responses from pictures specifically when the objects are actable (Bushong et al, 2010;Gomez et al, 2018;Fairchild et al, 2021;Snow and Culham, 2021;Snow et al, 2023).…”

“…Multipronged evidence from studies of behavior suggests that real objects are processed differently than pictures. For example, compared to 2-D pictures, real objects facilitate perception (Romero et al, 2018;Holler et al, 2020) and recognition (Ratcliff and Newcombe, 1982;Riddoch and Humphreys, 1987;Chainay and Humphreys, 2001;Farah, 2004;Turnbull et al, 2004;Holler et al, 2019), attract attention (Mustafar et al, 2015;Gerhard et al, 2016;Gomez et al, 2018;Korisky et al, 2019;Korisky and Mudrik, 2021;Sensoy et al, 2021), enhance memory (Snow et al, 2014(Snow et al, , 2023, increase valuation (Bushong et al, 2010;Romero et al, 2018), and modulate executive function (Beaucage et al, 2020) and social cognition (Kingstone, 2009;Pönkänen et al, 2011;Risko et al, 2012;Kuhn et al, 2016).…”

“…Importantly, real objects elicit different responses from both 2-D and 3-D stereoscopic pictures (Gomez et al, 2018;Holler et al, 2019Holler et al, , 2020Snow et al, 2023), indicating that response differences across formats cannot be explained by differences in the availability of stereoscopic depth cues (Snow and Culham, 2021). Nevertheless, depth cues provide a gateway to other critical object characteristics, including physical distance and real-world size.…”

“…Importantly, egocentric distance and real-world size determine whether a real object is actable: Only appropriately sized objects positioned within reach can be grasped. Accordingly, behavioral and EEG studies show that responses to real objects are influenced by egocentric distance (Gomez et al, 2018;Snow et al, 2023) and actability (Bushong et al, 2010;Fairchild et al, 2021), as well as real-world size (Holler et al, 2019(Holler et al, , 2020Sensoy et al, 2021), whereas responses to pictures are not. Despite accumulating evidence that display format and stimulus distance combine to influence behavior, none of the neuroimaging studies directly comparing real objects and pictures (Fairchild et al, 2021;Freud et al, 2018;Marini et al, 2019;Snow et al, 2011) have examined the effect of distance on object representations.…”