Sparse coding with population sketches

Abstract

Sparsity plays two very important roles in sensory neural information processing. At the cellular level, the sparse activation of cells within a population aids in minimizing the total metabolic cost required to represent sensory stimuli. At the population level, sparse codes provide a concise representation of complex stimuli, enabling efficient learning and storage/recall of memories. In models for sparse coding in the primary visual cortex (V1), the receptive fields of simple cells in the input layer of V1 serve as the basis from which a sparse representation of the visual environment is constructed. Recently, a number of neurally plausible mechanisms for sparse coding were developed [1], yielding testable predictions of cortico-cortical interactions of cells within a population by inducing local competition to minimize the total number of active cells in the network. Although this algorithmic framework provides an explicit mechanism for simple cell sparse coding, many still contend that current models for sparse coding are not plausible due to the overhead required to transmit both the locations and spike rates of all active cells within the population to higher areas in the cortex.With this in mind, we set out to develop a novel model for information processing in V1 that encourages sparse responses at the cellular level while also producing a sparse distributed representation that may be easily passed on to higher cortical areas. Taking a number of studies of information flow and connectivity into account, we have developed a new mathematical framework that solves connectivity constrained sparse coding problems; in this setting, a number of distinct computational units are presented the same input and must independently approximate the incoming signal and then efficiently minimize the number of these units needed to produce the final sparse representation. Our proposed sparse coding framework is inspired by the organization of simple cells with common receptive field features into densely connected networks known as minicolumns. In addition to minimizing long-range connections, we hypothesized that neural systems may take advantage of columnar connectivity constraints by computing a highlevel sketch of each microcircuit's response to an impending stimulus. Any model that describes the computation of this distributed population sketch must ensure that: "information" needed for complex inference tasks is preserved and that these tasks be performed without requiring knowledge of the single cell responses underlying the observed population activity.The most natural choice for the structures that will serve as the basis of our new representation, are the orientation minicolumns in V1. These densely connected networks consist of a number of simple cells tuned to the same orientation as well as other types of excitatory cells and interneurons. Connecting these distributed microcircuits is a small network of cells in layers II/III that we will refer to as a fusion network. The function of this network is still not...