Separating memoranda in depth increases visual working memory performance

Abstract

Visual working memory is the mechanism supporting the continued maintenance of information after sensory inputs are removed. Although the capacity of visual working memory is limited, memoranda that are spaced farther apart on a 2D display are easier to remember, potentially because neural representations are more distinct within retinotopically-organized areas of visual cortex during memory encoding, maintenance, and/or retrieval. The impact of spatial separability in depth on memory is less clear, even though depth information is essential to guide interactions with objects in the environment. On one account, separating memoranda in depth may facilitate performance if interference between items is reduced. However, depth information must be inferred indirectly from the 2D retinal image, and less is known about how visual cortex represents depth. Thus, an alternative possibility is that separation in depth does not attenuate between-item interference; separation in depth may even impair performance, as attention must be distributed across a larger volume of 3D space. We tested these alternatives using a stereo display while participants remembered the colors of stimuli presented either near or far in the 2D plane or in depth. Increasing separation in-plane and in depth both enhanced performance. Furthermore, participants who were better able to utilize stereo depth cues showed larger benefits when memoranda were separated in depth, particularly for large memory arrays. The observation that spatial separation in the inferred 3D structure of the environment improves memory performance, as is the case in 2D environments, suggests that separating memoranda in depth might reduce neural competition by utilizing cortically separable resources.

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

“…(1) In visual working memory studies, researchers often call the old/new paradigm "change detection", and quantify memory capacity using "K" values [N * (hits -false alarms), where N is the number of objects shown] (Cowan, 2001; see also Pashler 1988) (e.g., Alvarez & Cavanagh, 2004;Alvarez & Cavanagh, 2008;Brady & Alvarez, 2015;Chunharas, Rademaker, Sprague, Brady & Serences, 2019;Endress & Potter, 2014;Eriksson, Vogel, Lansner, Bergstrom, & Nyberg, 2015;Fukuda & Vogel, 2019;Fukuda, Vogel, Mayr & Awh, 2010;Fukuda, Woodman, & Vogel, 2015;Fukuda, Kang & Woodman, 2016;Hakim, Adam, Gunseli, Awh & Vogel, 2019;Irwin, 2014;;Pailian, Simons, Wetherhold, & Halberda, 2020;Schurgin & Brady, 2019;Shipstead, Lindsey, Marshall, & Engle, 2014;Sligte, Scholte, & Lamme, 2008;Unsworth, Fukuda, Awh, & Vogel, 2014;Unsworth, Fukuda, Awh, & Vogel, 2015;Vogel & Machizawa, 2004;Woodman & Vogel, 2008). A common assumption is that hits minus false alarms "corrects" for guessing, and thus 'K' measures "how many" items are remembered.…”

Increasing perceptual separateness affects working memory for depth – re-allocation of attention from boundaries to the fixated center

“…Research also suggests that cognitive factors, such as attention ( Griffin & Nobre, 2003 ; Murray, Nobre, Clark, Cravo, & Stokes, 2013 ; Qian, Zhang, Lei, Han, & Li, 2020 ), familiarity ( Olson & Poom, 2005 ), spatial configuration ( Li, Qian, & Liang, 2018 ; Jiang, Makovski, & Shim, 2008 ; Jiang, Olson, & Chun, 2000 ), and long-term memory ( Brady, Störmerb, & Alvarez, 2016 ), may help to overcome the capacity limitations of vWM. In addition, several studies have shown that when memory items were separated in different depth surfaces or planes, vWM performance was improved ( Chunharas, Rademaker, Sprague, Brady, & Serences, 2019 ; Xu & Nakayama, 2007 ), especially for those objects closer to the observer ( Qian, Li, Wang, Liu, & Lei, 2017 ; Qian, Zhang, Wang, Li, & Lei, 2018 ).…”

“…If presenting items in separate depth planes facilitates the memory performance for their visual features ( Chunharas, Rademaker, Sprague, Brady, & Serences, 2019 ; Qian et al, 2017 ; Qian et al, 2018 ), one may suspect if the reverse is true – whether separating items by different visual features could improve the memory performance for their depth locations. The memory benefits of depth separation may arise from a process of reducing neural competition by utilizing cortically separable resources ( Chunharas et al, 2019 ), and thus facilitating the encoding of visual features ( Nakayama & Silverman, 1986 ).…”

Visual Search in 3D: Effects of Monoscopic and Stereoscopic Cues to Depth on the Validity of Feature Integration Theory and Perceptual Load Theory

“…Yet, in a similar paradigm, Pomplun et al (2013) report no substantial effects of depth of plane on visual search strategy derived from eye movement data. Other work has demonstrated that the apparent proximity of stimuli affects visual tracking ( Viswanathan and Mingolla, 2002 ) and visual working memory ( He and Nakayama, 1995 ; Xu and Nakayama, 2007 ; Qian et al, 2017 , 2018 ; Chunharas et al, 2019 ). Finally a third body of work has explored visual search and foraging performance in dynamic virtual reality 3D environments ( Prpic et al, 2019 ; Kristjánsson et al, 2020 ), sometimes compared to traditional 2D pictorial scenes ( Li et al, 2016 ).…”