Transsaccadic integration relies on a limited memory resource

Abstract

Saccadic eye movements cause large-scale transformations of the image falling on the retina. Rather than starting visual processing anew after each saccade, the visual system combines post-saccadic information with visual input from before the saccade. Crucially, the relative contribution of each source of information is weighted according to its precision, consistent with principles of optimal integration. We reasoned that, if pre-saccadic input is maintained in a resource-limited store, such as visual working memory, its precision will depend on the number of items stored, as well as their attentional priority. Observers estimated the color of stimuli that changed imperceptibly during a saccade, and we examined where reports fell on the continuum between pre- and post-saccadic values. Bias toward the post-saccadic color increased with the set size of the pre-saccadic display, consistent with an increased weighting of the post-saccadic input as precision of the pre-saccadic representation declined. In a second experiment, we investigated if transsaccadic memory resources are preferentially allocated to attentionally prioritized items. An arrow cue indicated one pre-saccadic item as more likely to be chosen for report. As predicted, valid cues increased response precision and biased responses toward the pre-saccadic color. We conclude that transsaccadic integration relies on a limited memory resource that is flexibly distributed between pre-saccadic stimuli.

Post-saccadic changes disrupt attended pre-saccadic object memory

“…Trans-saccadic memory has been considered to be based on visual working memory ( Luck & Vogel, 1997 ), due to the similarities in limited memory capacity as well as the timing duration of a few seconds ( Bays & Husain, 2008 ; Cowan, 2011 ; Irwin, 1992 ; Irwin, 1996 ; Jeyachandra et al, 2018 ; Luck & Vogel, 1997 ). Previous studies have shown evidence for automatic attention leading to automatic encoding into visual working/trans-saccadic memory ( Hollingworth & Matsukura, 2019 ; Jonikaitis & Moore, 2019 ; Kong, Kroell, Schneegans, Aagten-Murphy, & Bays, 2021 ), at the expense of the item already in memory, namely here memory of the pre-saccadic symbol is disrupted, resulting in decreased discrimination performance. The observed disruption of trans-saccadic memory due to post-saccadic changes therefore adds arguments for limited resources in trans-saccadic memory ( Poth et al, 2015 ).…”

Transsaccadic integration operates independently in different feature dimensions

“…However, we would argue this is unlikely. A comparison between the results of Experiments 2 and 3 indicates that attention is not critical in determining whether the integration process is feature based, and another recent study (Kong et al, 2021) demonstrated that visual working memory is involved in the integration process for multiple stimuli, not just the saccade target.…”

“…Recently, the term serial dependence has been used to describe phenomena in which the report of a visual stimulus is influenced by a statistically unrelated stimulus presented earlier, typically on the previous trial (Fischer & Whitney, 2014). Like transsaccadic integration (Irwin, 1991;Kong et al, 2021;Oostwoud Wijdenes et al, 2015;Schut et al, 2017), it has been suggested that serial dependence involves an integration of information obtained at two different points of time based on representations in working memory (Bliss, Sun, & D'Esposito, 2017;Fritsche, Mostert, & de Lange, 2017). Importantly for the purposes of this discussion, serial dependence also appears to happen on the level of features.…”

“…This makes integrating pre-and postsaccadic visual information a potentially advantageous strategy, as the combined estimate should be less influenced by perceptual noise. However, as the fidelity of presaccadic information available for transsaccadic computations is limited (Kong, Kroell, Schneegans, Aagten-Murphy, & Bays, 2021), any evidence for a mismatch with the current environment could be considered justification to discount it and rely exclusively on the current (postsaccadic) input. However, across three experiments, we have found that the visual system operates according to different rules.…”

“…Citation: Kong, G., Aagten-Murphy, D., McMaster, J. M. V., & Bays, P. M. (2021). Transsaccadic integration operates independently in different feature dimensions.…”