Keeping it real: Looking beyond capacity limits in visual cognition

Abstract

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate “for free” and “on the fly.” These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Visual working memory for connected 3D objects: effects of stimulus complexity, dimensionality and connectivity

“…Third, while dimensionality and connectivity do not enhance detection of a color change, they might affect the detection of other changes, such as changes to the binding of color to structure, as in structural isomers which are composed of the same atoms but in a different configuration. Finally, the visual working memory paradigm may underestimate people’s capacities compared to more naturalistic tasks that scientists engage in, which do not require maintaining the complete representation of a complex stimulus in memory (Kristjansson & Draschkow, 2021 ).…”

“…These experiments help to fill a gap in the current VWM literature. To date, research has examined VWM for separated two-dimensional visual units (e.g., Brady & Alvarez, 2015b ; Brady et al, 2011 ; Luck & Vogel, 1997 ), separated three-dimensional objects (Brady & Alvarez, 2015a ; Wood, 2011 ) and real-world objects (Brady et al, 2009b ; Kristjansson & Draschkow, 2021 ). Inspired by STEM representations, the present work extends this research to complex multicomponent stimuli and rules out a possible advantage for 3D object representations in detecting color changes, moving the field a step closer to understanding how VWM works for complex STEM representations.…”

Advances in the application of a computational Theory of Visual Attention (TVA): Moving towards more naturalistic stimuli and game-like tasks

“…In this section we review recent trends towards taking TVA-based research out of laboratory settings. The goal behind this is twofold: (1) moving outside the lab can make the method more accessible to various populations of interest and (2) bringing tasks into behaviorally richer environments such as games and virtual reality enables studying attention under more ecologically valid conditions (Kristjánsson & Draschkow, 2021). These approaches can also be combined.…”

“…While tasks with discrete trial presentation and reactions can certainly provide useful information about attentional orienting, their main drawback is that they involve a reductionist approach, where the aim is to measure attention allocation in a "pure" sense, in isolation from other processes. Some authors have argued that more dynamic and continuous tasks may therefore better reflect actual attention allocation in real world scenarios (Kristjánsson & Draschkow, 2021;Hayhoe, 2017). In a recent study on selective attention in visual search tasks, Sauter, Stefani, and Mack (2020) lists several points in which searching for objects in the real world differs from searching for predefined targets as studied in many laboratory search tasks.…”

“…The majority of TVA-based research is based on accuracy data from letter report experiments, a task-model combination commonly applied to estimate attentional parameters in clinical contexts. While such basic approaches are undoubtedly useful, recent technological advances have enabled the testing of increasingly sophisticated paradigms involving more realistic scenarios and notably those studies have in many cases revealed different capacity limits than studies using simple, but well controlled stimuli (Kristjánsson & Draschkow, 2021). Virtual Reality experiments allow immersion in "real-world" like settings and touch-screen technology allows direct interaction with the stimuli.…”