Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening

Pehlevan, Cengiz; Chklovskii, Dmitri B.

doi:10.1109/allerton.2015.7447180

Cited by 20 publications

(23 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a mechanism appears useful to attenuate the impact of irrelevant sensory inputs and to reduce undesired correlations. The mechanism of whitening by feature suppression is consistent with networks that have been optimized for whitening in a theoretical framework with biologically plausible constraints (Pehlevan and Chklovskii, 2015; Zung and Seung, 2017). Hence, the mechanism of whitening observed in the OB may represent a general computational strategy in the brain.…”

Section: Discussionsupporting

confidence: 68%

Whitening of odor representations by the wiring diagram of the olfactory bulb

Wanner

Friedrich

2019

Preprint

View full text Add to dashboard Cite

One of the fundamental questions in neuroscience is how computational functions of neuronal 1 circuits are determined by their connectivity. The olfactory bulb decorrelates and normalizes 2 patterns of sensory input, resulting in a transformation referred to as whitening that supports 3 pattern classification. To analyze the underlying mechanisms we measured odor-evoked activity 4 throughout the OB of a zebrafish larva and reconstructed the wiring diagram in the same specimen 5 by volume electron microscopy. This functional connectomics approach revealed an 6 overrepresentation of triplet connectivity motifs that privilege disynaptic reciprocal inhibition 7 among co-tuned output neurons. This connectivity resulted in strong inhibition of neuronal 8 responses to common features of correlated sensory input patterns. In generic network models, this 9 connectivity was necessary and sufficient to reproduce whitening of naturalistic inputs. These 10 results reveal a specific network topology that is adapted to the statistical structure of olfactory 11 inputs and mediates whitening by feature suppression. 12

show abstract

Section: Discussionsupporting

confidence: 68%

Whitening of odor representations by the wiring diagram of the olfactory bulb

Wanner

Friedrich

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Various versions of Hebbian plasticity (Miller and MacKay, 1994), e.g., with nonlinearities (Brito and Gerstner, 2016), can give rise to different forms of correlation and competition between neurons, leading to the self-organized formation of ocular dominance columns, selforganizing maps and orientation columns (Ferster and Miller, 2003;Miller et al, 1989). Often these types of local self-organization can also be viewed as optimizing a cost function: for example, certain forms of Hebbian plasticity can be viewed as extracting the principal components of the input, which minimizes a reconstruction error (Pehlevan and Chklovskii, 2015).…”

Section: Local Self-organization and Optimization Without Multi-layermentioning

confidence: 99%

Towards an integration of deep learning and neuroscience

Marblestone

Wayne

Körding

2016

Preprint

View full text Add to dashboard Cite

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short-and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) these cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.

show abstract

“…The novelty of our approach is that the algorithm is derived from the similarity matching principle which postulates that neural circuits map more similar inputs to more similar outputs (Pehlevan et al, 2015). Previous work proposed various objective functions to find similarity matching neural representations and solved these optimization problems with biologically plausible neural networks (Pehlevan et al, 2015;Chklovskii, 2015a, 2014;Hu et al, 2014;Pehlevan and Chklovskii, 2015b). Here we apply these networks to NICA.…”

Section: Introductionmentioning

confidence: 99%

Blind Nonnegative Source Separation Using Biological Neural Networks

2017

Self Cite

View full text Add to dashboard Cite

Blind source separation, i.e. extraction of independent sources from a mixture, is an important problem for both artificial and natural signal processing. Here, we address a special case of this problem when sources (but not the mixing matrix) are known to be nonnegative, for example, due to the physical nature of the sources. We search for the solution to this problem that can be implemented using biologically plausible neural networks. Specifically, we consider the online setting where the dataset is streamed to a neural network. The novelty of our approach is that we formulate blind nonnegative source separation as a similarity matching problem and derive neural networks from the similarity matching objective. Importantly, synaptic weights in our networks are updated according to biologically plausible local learning rules.

show abstract

Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening

Cited by 20 publications

References 23 publications

Whitening of odor representations by the wiring diagram of the olfactory bulb

Whitening of odor representations by the wiring diagram of the olfactory bulb

Towards an integration of deep learning and neuroscience

Blind Nonnegative Source Separation Using Biological Neural Networks

Contact Info

Product

Resources

About