Temporal and spatial reference frames in visual working memory are defined by ordinal and relational properties.

Abstract

Natural environments provide a rich spatiotemporal context that allows for visual objects to be differentiated based on different types of information: their absolute or relative spatial or temporal coordinates, or their ordinal positions in a spatial or temporal sequence. Here, we investigated which spatial and temporal properties are incidentally encoded along with to-be-remembered features to provide reference frames in visual working memory (VWM). We tested the different possibilities in a spatiotemporal color change-detection task by transforming spatial and/or temporal structures of item presentation at retrieval relative to encoding. More precisely, spatial and/or temporal coordinates were (a) switched, changing the order of items in a spatial or temporal sequence (ordinal transformation); (b) multiplied by different factors, changing interitem distances (relational transformation); or (c) multiplied by a constant factor, expanding or shrinking the entire configuration (global transformation). Such transformations of the external reference frame at retrieval should only interfere with VWM if the internal reference frame relies on the spatial or temporal properties affected by the respective transformation. We found that ordinal and relational transformations of either the spatial or temporal structure impaired performance, whereas global transformations did not. Thus, reference frames appear to be primarily defined by interitem relations-including relative distances between items as well as their order-rather than absolute positions in space or time. These results corroborate and extend previous findings for the spatial domain, and highlight functional similarities of the spatial and temporal dimensions in VWM by revealing the same metrical properties for temporal reference frames.

Capacity and selection in immersive visual working memory following naturalistic object disappearance

“…in front or behind). These observations build on complementary studies pointing to the availability and utilisation of allocentric codes for working memory [27,42,43], including following self-movement [27] as opposed to object movement that we studied here.…”

Sensory Input Matching Visual Working Memory Guides Internal Prioritization

“…less costs of cueing), having relatively strong biasing effects on reports of cued items, or even improving long-term memory representations. Other open questions concern how other types of information that dynamic sensory input can convey such as temporal order and (un)predictable motion may be used to guide VWM prioritization (as in Figure 1; also see Heuer & Rolfs, 2023).…”

“…(un)predictable motion may be used to guide VWM prioritization (as in Figure 1; also see Heuer & Rolfs, 2023).…”

“…As van Ede and Nobre (2023) argue, many retro cue experiments may not reflect how memorized material is prioritized in natural settings, as one seldom receives explicit spatial or feature-based cues indicating what VWM content is most relevant. One way that memories may be prioritized more naturally is based on temporal expectations (Heuer & Rolfs, 2023; Nobre & van Ede, 2023; van Ede et al, 2017). Being able to predict when to act upon a specific representation allows to prioritize that representation which enhances its fidelity and protects it from interference (Gresch et al, 2022; Gresch et al, 2021; van Ede et al, 2017).…”

Capacity and selection in immersive visual working memory following naturalistic object disappearance

“…In future studies it could be interesting to vary the visual flow of objects in such a way that it would also be possible to disentangle multiple frames of references (e.g., allocentric and egocentric) and investigate which spatial frames are used for selection (cf. Draschkow et al, 2022 ; Heuer & Rolfs, 2022 ; Klinghammer, Schütz, Blohm, & Fiehler, 2016 ; Pertzov, Avidan, & Zohary, 2011 ) following naturalistic object disappearance in such a context.…”