Representation and computation in working memory

Abstract

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have driven significant advances in our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event, as new experimental tools have revealed that observers possess richer information about the uncertainty in their memories, and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visual WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research in human visual WM from a computational perspective and the latest developments in identifying neural mechanisms that support it.

Mapping visual working memory models to a theoretical framework

“…Despite finding evidence for guessing responses that cannot straightforwardly be explained by a noisy memory, the authors noted that the finding does not determine whether VWM resources are discrete or continuous in nature -the observed data could be explained by a version from both discrete and continuous classes of models, if they are allowed some modifications. Indeed, this is a usual retort by proponents of pure resource-based or resource-rational models -that modern resource models can account for such zero-precision estimates without an additional mechanism (Bays et al, 2022;Schneegans et al, 2020;van den Berg & Ma, 2018). Then, it is unclear what data and what evidence would be needed to deterministically decide between existing models of VWM.…”

“…It may also provide a common language or platform for researchers to discuss their varying perspectives on how the VWM system operates. VWM models have vastly and rapidly evolved since their initial slot or resource conceptions, and the wider field may not have kept track of the key data and subsequent critical changes to the models (Bays et al, 2022). For example, despite the strong object model generally being disproven (Hardman & Cowan, 2015;Olson & Jiang, 2002) and not currently widely believed, its specification and ideas continue to feature in research in various ways (see Robinson et al, 2022;Williams et al, 2022).…”

“…A core tenet of resource models is that working memory representations vary in strength in a continuous manner. I have straightforwardly mapped this on to the binding pool nodes to encapsulate that facets of VWM like variable precision has been predominantly modeled as noise in neural populations (Bays, 2014;Bays et al, 2022;Schneegans et al, 2020;Schneegans & Bays, 2017). Population coding accounts model the representation of VWM as an encoding-decoding process -a variable number of samples generated from a neural population tuned for feature values is read-out to produce a signal that informs response behavior (Bays, 2014;Schneegans et al, 2020).…”

A dynamic neural resource model bridges sensory and working memory

“…Keeping relevant information in an easily accessible state is vital for adaptive behavior in dynamic environments. In the primate visual system, this requirement is met by visual working memory (VWM), the capacity to actively maintain visual information from milliseconds to seconds after a stimulus disappears from view [1][2][3][4]. While the contents of VWM are frequently updated to reflect changes in the environment and in behavioral priorities, the visual processing hierarchy itself introduces additional layers of dynamism [5,6].…”

Visual working memory phenomena based on categorical tasks replicate using a continuous measure: A simple interpretation and some methodological considerations

“…One is earlier work principally concerned with its role in visual imagery, complex processing and long-term memory (Baddeley & Lieberman, 1980;Logie, 1986;Shepard & Metzler, 1971), although a notable exception to this was the work of Phillips (Phillips, 1974;Phillips & Baddeley, 1971;Phillips & Christie, 1977) that was a direct forerunner of the currently dominant approach to visual working memory. This more recent approach has stemmed from research on visual attention and focuses on a more detailed understanding of the earlier stages of visual memory (e.g., Bays et al, 2009;Kahneman et al, 1992;Luck & Vogel, 1997;Wheeler & Treisman, 2002;Zhang & Luck, 2008; for a review see Bays et al, 2022). Our own work in recent years has attempted to bridge the two fields by applying methods that were originally developed for verbal material to their visual equivalent (for reviews, see Baddeley et al, 2011a, and Hitch et al, 2020.…”