Sparse coding of sensory inputs

Olshausen, Bruno A.; Field, David

doi:10.1016/j.conb.2004.07.007

Cited by 1,220 publications

(977 citation statements)

References 51 publications

Supporting

Mentioning

894

Contrasting

Unclassified

Order By: Relevance

“…However, these approaches rarely take into account the animal's tasks and goals. This presents a problem because, in addition to representing stimuli as faithfully and efficiently as possible, nervous systems must also reduce information to facilitate downstream computations that inform behavior (Barlow, 2001;Olshausen and Field, 2004). In support of this, many sensory codes are often not "optimal" in the classical sense (Salinas, 2006).…”

Section: Introductionmentioning

confidence: 99%

Connecting Neural Codes with Behavior in the Auditory System of Drosophila

Clemens¹,

Girardin²,

Coen³

et al. 2018

Neuron

View full text Add to dashboard Cite

SummaryBrains are optimized for processing ethologically relevant sensory signals. However, few studies have characterized the neural coding mechanisms that underlie the transformation from natural sensory information to behavior. Here, we focus on acoustic communication in Drosophila melanogaster, and use computational modeling to link natural courtship song, neuronal codes, and female behavioral responses to song. We show that melanogaster females are sensitive to long timescale song structure (on the order of tens of seconds). From intracellular recordings, we generate models that recapitulate neural responses to acoustic stimuli. We link these neural codes with female behavior by generating model neural responses to natural courtship song. Using a simple decoder, we predict female behavioral responses to the same song stimuli with high accuracy. Our modeling approach reveals how long timescale song features are represented by the Drosophila brain, and how neural representations can be decoded to generate behavioral selectivity for acoustic communication signals.

show abstract

Section: Introductionmentioning

confidence: 99%

Connecting Neural Codes with Behavior in the Auditory System of Drosophila

Clemens¹,

Girardin²,

Coen³

et al. 2018

Neuron

View full text Add to dashboard Cite

show abstract

“…The potential degradation of selectivity by saturation (Robson, 1975) is largely prevented by a contrast gain control in striate cortex, which preserves the CRF shape for nonoptimal stimuli (Albrecht & Hamilton, 1982;Skottun et al, 1987;Albrecht & Geisler, 1991;Bonds, 1991;Geisler & Albrecht, 1992;Carandini & Heeger, 1994). The CRF shape also has theoretical implications for efficiency of sensory coding (Simoncelli, 2003), both for enhancement of sparse coding (Olshausen & Field, 2004) and for ensuring efficient distribution of firing rates in response to natural images (Laughlin, 1981;Baddeley et al, 1998).…”

Section: Introductionmentioning

confidence: 99%

The direction-selective contrast response of area 18 neurons is different for first- and second-order motion

et al. 2005

View full text Add to dashboard Cite

Cortical neurons selective for the direction of motion often exhibit some limited response to motion in their nonpreferred directions. Here we examine the dependence of neuronal direction selectivity on stimulus contrast, both for first-order (luminance-modulated, sine-wave grating) and second-order (contrast-modulated envelope) stimuli. We measured responses from single neurons in area 18 of cat visual cortex to both kinds of moving stimuli over a wide range of contrasts (1.25-80%). Direction-selective contrast response functions (CRFs) were calculated as the preferred-minus-null difference in average firing frequency as a function of contrast. We also applied receiver operating characteristic analysis to our CRF data to obtain neurometric functions characterizing the potential ability of each neuron to discriminate motion direction at each contrast level tested. CRFs for sine-wave gratings were usually monotonic; however, a substantial minority of neurons (35%) exhibited nonmonotonic CRFs (such that the degree of direction selectivity decreased with increasing contrast). The underlying preferred and nonpreferred direction CRFs were diverse, often having different shapes in a given neuron. Neurometric functions for direction discrimination showed a similar degree of heterogeneity, including instances of nonmonotonicity. For contrast-modulated stimuli, however, CRFs for either carrier or envelope contrast were always monotonic. In a given neuron, neurometric thresholds were typically much higher for second-than for first-order stimuli. These results demonstrate that the degree of a cell's direction selectivity depends on the contrast at which it is measured, and therefore is not a characteristic parameter of a neuron. In general, contrast response functions for first-order stimuli were very heterogeneous in shape and sensitivity, while those for second-order stimuli showed less sensitivity and were quite stereotyped in shape.

show abstract

“…The CBCL face dataset is extensively used by the MIT Center For Biological and Computation Learning to test the performance of face-detection systems 4 . The dataset provides 19×19 gray-scale images including 2.429 faces for training and 472 faces for testing.…”

Section: Dataset and Network Trainingmentioning

confidence: 99%

“…network parameters and states are constrained to be positive only. Beside non-negativity, neural physiological evidence as well as energy constraints [4] motivate to encode sensory inputs with only a few neurons active at a specific point in time. This strategy of encoding is commonly referred to as sparse coding.…”

Section: Introductionmentioning

confidence: 99%

“…This strategy of encoding is commonly referred to as sparse coding. Describing patterns with less active neurons minimizes the probability of destructive cross talk [4] and enhances the probability of linear separability, a crucial feature for object classification [5]. To address these requirements, several matrix factorization approaches have been developed that decompose a set of patterns into parts and their configuration which both are non-negative and sparse.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online learning and generalization of parts-based image representations by non-negative sparse autoencoders

2012

View full text Add to dashboard Cite

We present an efficient online learning scheme for non-negative sparse coding in autoencoder neural networks. It comprises a novel synaptic decay rule that ensures non-negative weights and an intrinsic self-adaptation rule that optimizes sparseness of the non-negative encoding. We show that non-negativity constrains the space of solutions such that overfitting is prevented and very similar encodings are found irrespective of the network initialization and size. We benchmark the novel method on real-world datasets of handwritten digits and faces. The autoencoder yields higher sparseness and lower reconstruction errors than related batch algorithms based on matrix factorization. It generalizes to new inputs both accurately and without costly computations, which is fundamentally different from the classical matrix factorization approaches.

show abstract

Sparse coding of sensory inputs

Cited by 1,220 publications

References 51 publications

Connecting Neural Codes with Behavior in the Auditory System of Drosophila

Connecting Neural Codes with Behavior in the Auditory System of Drosophila

The direction-selective contrast response of area 18 neurons is different for first- and second-order motion

Online learning and generalization of parts-based image representations by non-negative sparse autoencoders

Contact Info

Product

Resources

About