Capacity and selection in immersive visual working memory following naturalistic object disappearance

Abstract

Visual working memory – holding past visual information ‘in mind’ for upcoming behaviour – is commonly studied following the abrupt removal of visual objects from static 2D displays. In everyday life, visual objects do not typically vanish from the environment in front of us. Rather, visual objects tend to enter working memory following self or object motion: disappearing from view gradually and changing the spatial relation between memoranda and observer. Here, we used virtual reality (VR) to investigate whether two classic findings from visual working memory research – a capacity of around three objects and the reliance on space for object selection – generalise to more naturalistic modes of object disappearance. Our static reference condition mimicked traditional laboratory tasks whereby visual objects were held static in front of the participant and removed from view abruptly. In our critical flow condition, the same visual objects flowed by participants, disappearing from view gradually and behind the observer. We considered visual working memory capacity as well as space-based mnemonic selection, indexed by directional biases in gaze. Despite vastly distinct modes of object disappearance and altered spatial relations between memoranda and observer, we found comparable capacity and comparable gaze signatures of space-based mnemonic selection. This reveals how classic findings from visual working memory research generalise to immersive situations with more naturalistic modes of object disappearance and with dynamic spatial relations between memoranda and observer.

Sparse spatial scaffolding for visual working memory

“…Ultimately, the strength of our inference hinges on the suitability of our spatial marker for studying spatial scaffolding for working memory. Our findings build directly on several prior studies where we successfully leveraged spatial biases in fixational gaze behaviour to study spatial scaffolding for working memory – without ever asking participants about memorised item location 15,19,25,34–36 . Here, we for the first time use this implicit marker to study the use of distance.…”

“…We unveil how the human brain engages fundamentally dis]nct spa]al codes for retaining visual representa]ons in working memory, depending on the u]lity of spa]al features as a scaffold for memory. Leveraging spa]al biases in fixa]onal gaze behaviour during mnemonic selec]on as an implicit read-out of spa]al scaffolding for visual working memory (as in 19,25,34,35 ), our present data unveil the principle of "sparse spa]al scaffolding" for visual working memory, whereby the brain uses the minimal spa]al features necessary for the separa]on and selec]on of individual contents in visual working memory.…”

“…1a). We presented items either near (3 degrees) or far (6 degrees) from the central fixa]on dot and used spa]al biases in fixa]onal gaze behaviour during mnemonic selec]on as an implicit read-out of the spa]al scaffolding used for visual working memory (an approach that we applied successfully in several recent complementary studies: 19,25,[34][35][36] ).…”

“…To study whether distance was incorporated in the spa]al scaffold used for working memory -in addi]on to direc]on -we collapsed lek and right trials into a single measure of 'towardness' (as in [34][35][36][37] ) and overlayed towardness between trials with cues to near and far items. As shown in Figure 1c, we found that gaze aker the retro-cue became significantly biased in the direc]on of the cued memory item, both when the cued item was near (cluster P < 0.001) or far (cluster P < 0.001).…”

“…Building on our complementary recent findings and approaches 19,25,34,35 , we leveraged spatial biases in fixational gaze behaviour during mnemonic selection as a read-out of spatial coding for visual working memory. This enabled us to probe spatial scaffolding for working memory implicitly, without ever asking participants about memorised locations.…”