Visual objects interact differently during encoding and memory maintenance

Abstract

The storage mechanisms of working memory are the matter of an ongoing debate. The sensory recruitment hypothesis states that memory maintenance and perceptual encoding rely on the same neural substrate. This suggests that the same cortical mechanisms that shape object perception also apply to maintained memory content. We tested this prediction using the Direction Illusion, i.e., the mutual repulsion of two concurrently visible motion directions. Participants memorized the directions of two random dot patterns for later recall. In Experiments 1 and 2, we varied the temporal separation of spatially distinct stimuli to manipulate perceptual concurrency, while keeping concurrency within working memory constant. We observed mutual motion repulsion only under simultaneous stimulus presentation, but proactive repulsion and retroactive attraction under immediate stimulus succession. At inter-stimulus intervals of 0.5 and 2 s, however, proactive repulsion vanished, while the retroactive attraction remained. In Experiment 3, we presented both stimuli at the same spatial position and observed a reappearance of the repulsion effect. Our results indicate that the repulsive mechanisms that shape object perception across space fade during the transition from a perceptual representation to a consolidated memory content. This suggests differences in the underlying structure of perceptual and mnemonic representations. The persistence of local interactions, however, indicates different mechanisms of spatially global and local feature interactions.

Visual working memory items drift apart due to active, not passive, maintenance.

“…While some prior studies have used a non-target item as a reference point (e.g., Chunharas et al, 2019;Czoschke et al, 2019), several other studies have used hierarchical properties like the ensemble statistic (e.g., Brady & Alvarez, 2011) or the average color observed across all past trials (e.g., Huang & Sekuler, 2010). We note that our results could differ depending on what property the memory system uses as the reference point for repulsion bias because hierarchical properties may be automatically computed and take up space in memory independently from concurrently maintained memory items (for discussion see Brady, Konkle, & Alvarez, 2011).…”

“…These perceptual illusions might be related to the attraction/repulsion biases observed in previous working memory studies. Previous working memory studies have used paradigms where items are encoded and tested in the same spatial location, such that working memory maintenance might have relied on retinotopically specific, sustained sensory activation (Czoschke, Peters, Rahm, Kaiser, & Bledowski, 2019). In other words, persistent neural firing in sensory areas after stimulus offset could induce visual adaptation effects similar to those observed in the aforementioned perceptual illusions.…”

Visual working memory items drift apart due to active, not passive, maintenance

“…While some prior studies have used a non-target item as a reference point (e.g., Czoschke et al, 2019), several other studies have used hierarchical properties like the ensemble statistic (e.g., Brady & Alvarez, 2011) or the average color observed across all past trials (e.g., Huang & Sekuler, 2010). We note that our results could differ depending on what property the memory system uses as the reference point for repulsion bias because hierarchical properties may be automatically computed and take up space in memory independently from concurrently maintained memory items (for discussion see Brady, Konkle, & Alvarez, 2011).…”

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

“…Recent studies have revealed robust short-term influences of prior stimulation in delayed-response and working-memory tasks (Bae & Luck, 2017; Cicchini, Anobile, & Burr, 2014; Cicchini, Mikellidou, & Burr, 2018; Czoschke, Fischer, Beitner, Kaiser, & Bledowski, 2019; Czoschke, Peters, Rahm, Kaiser, & Bledowski, 2020; Fischer & Whitney, 2014; Fritsche, Mostert, & de Lange, 2017). In these studies, perceptual decision making is systematically influenced by stimulus features presented earlier in the trial (Bae & Luck, 2017; Czoschke, Fischer, Beitner, Kaiser, & Bledowski, 2019; Czoschke, Peters, Rahm, Kaiser, & Bledowski, 2020; Fritsche & de Lange, 2019; Fritsche, Mostert, & de Lange, 2017), in the previous trial (Cicchini, Mikellidou, & Burr, 2017; Cicchini, Mikellidou, & Burr, 2018; Fritsche, Mostert, & de Lange, 2017; Makovski & Jiang, 2008), or multiple trials back (Fritsche, Solomon, & De Lange, 2021; Fritsche, Spaak, & de Lange, 2020; Gekas, McDermott, & Mamassian, 2019; Suárez-Pinilla, Seth, & Roseboom, 2018). More specifically, items are reported as more similar to the task-relevant stimulus on a previous trial.…”

“…Similarly, if the attractive between-trial bias originates at the sensory processing stage, a shift in the neural representation would be expected to occur towards the target orientation on the previous trial. Based on previous research (Bae & Luck, 2020; Fischer, Czoschke, et al, 2020), we expected that between-trial biases would be modulated by the task-relevance of the item on the previous trial, such that shifts in representation would be restricted to or significantly larger for previously task-relevant items.…”