Experimental evidence suggests that the content of a memory for even a simple display encoded in visual short-term memory (VSTM) can be very complex. VSTM uses organizational processes that make the representation of an item dependent on the feature values of all displayed items as well as on these items' representations. Here, we develop a probabilistic clustering theory (PCT) for modeling the organization of VSTM for simple displays. PCT states that VSTM represents a set of items in terms of a probability distribution over all possible clusterings or partitions of those items. Because PCT considers multiple possible partitions, it can represent an item at multiple granularities or scales simultaneously. Moreover, using standard probabilistic inference, it automatically determines the appropriate partitions for the particular set of items at hand and the probabilities or weights that should be allocated to each partition. A consequence of these properties is that PCT accounts for experimental data that have previously motivated hierarchical models of VSTM, thereby providing an appealing alternative to hierarchical models with prespecified, fixed structures. We explore both an exact implementation of PCT based on Dirichlet process mixture models and approximate implementations based on Bayesian finite mixture models. We show that a previously proposed 2-level hierarchical model can be seen as a special case of PCT with a single cluster. We show how a wide range of previously reported results on the organization of VSTM can be understood in terms of PCT. In particular, we find that, consistent with empirical evidence, PCT predicts biases in estimates of the feature values of individual items and also predicts a novel form of dependence between estimates of the feature values of different items. We qualitatively confirm this last prediction in 3 novel experiments designed to directly measure biases and dependencies in subjects' estimates.

Sequential and persistent activity models are two prominent models of short-term memory in neural circuits. In persistent activity models, memories are represented in persistent or nearly persistent activity patterns across a population of neurons, whereas in sequential models, memories are represented dynamically by a sequential pattern of activity across the population. Experimental evidence for both types of model in the brain has been reported previously. However, it has been unclear under what conditions these two qualitatively different types of solutions emerge in neural circuits. Here, we address this question by training recurrent neural networks on several short-term memory tasks under a wide range of circuit and task manipulations. We show that sequential and nearly persistent solutions are both part of a spectrum that emerges naturally in trained networks under different conditions. Fixed delay durations, tasks with higher temporal complexity, strong network coupling, motion-related dynamic inputs and prior training in a different task favor more sequential solutions, whereas variable delay durations, tasks with low temporal complexity, weak network coupling and symmetric Hebbian short-term synaptic plasticity favor more persistent solutions. Our results help clarify some seemingly contradictory experimental results on the existence of sequential vs. persistent activity based memory mechanisms in the brain..

Sequential and attractor-based models are two prominent models of short-term memory in neural circuits. In attractor-based models, memories are represented in persistent or nearly persistent activity patterns across a population of neurons, whereas in sequential models, memories are represented dynamically by a sequential pattern of activity across the population. Experimental evidence for both types of model in the brain has been reported previously. However, it has been unclear under what conditions these two qualitatively different types of solutions emerge in neural circuits. Here, we address this question by training recurrent neural networks on several short-term memory tasks under a wide range of circuit and task manipulations. We show that fixed delay durations, tasks with higher temporal complexity, strong network coupling and motion-related dynamic inputs favor sequential solutions, whereas variable delay durations, tasks with low temporal complexity, weak network coupling and symmetric Hebbian short-term synaptic plasticity favor more persistent solutions. Our results clarify some seemingly contradictory experimental results on the existence of sequential vs. attractor-like memory mechanisms in the brain.

Animals perform near-optimal probabilistic inference in a wide range of psychophysical tasks. Probabilistic inference requires trial-to-trial representation of the uncertainties associated with task variables and subsequent use of this representation. Previous work has implemented such computations using neural networks with hand-crafted and task-dependent operations. We show that generic neural networks trained with a simple error-based learning rule perform near-optimal probabilistic inference in nine common psychophysical tasks. In a probabilistic categorization task, error-based learning in a generic network simultaneously explains a monkey’s learning curve and the evolution of qualitative aspects of its choice behavior. In all tasks, the number of neurons required for a given level of performance grows sublinearly with the input population size, a substantial improvement on previous implementations of probabilistic inference. The trained networks develop a novel sparsity-based probabilistic population code. Our results suggest that probabilistic inference emerges naturally in generic neural networks trained with error-based learning rules.

A growing body of scientific evidence suggests that visual working memory and statistical learning are intrinsically linked. Although visual working memory is severely resource limited, in many cases, it makes efficient use of its available resources by adapting to statistical regularities in the visual environment. However, experimental evidence also suggests that there are clear limits and biases in statistical learning. This raises the intriguing possibility that performance limitations observed in visual working memory tasks can to some degree be explained in terms of limits and biases in statistical-learning ability, rather than limits in memory capacity.