A neuron as a signal processing device

Abstract

Abstract-A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of synaptic weights scaled by an outgoing sparse activity vector. Formally, a neuron minimizes a cost function comprising a cumulative squared representation error and regularization terms. We derive an online algorithm that minimizes such cost function by alternating between the minimization with respect to activity and with respect to synaptic weights. The steps of this algorithm reproduce well-known physiological properties of a neuron, such as weighted summation and leaky integration of synaptic inputs, as well as an Oja-like, but parameter-free, synaptic learning rule. Our theoretical framework makes several predictions, some of which can be verified by the existing data, others require further experiments. Such framework should allow modeling the function of neuronal circuits without necessarily measuring all the microscopic biophysical parameters, as well as facilitate the design of neuromorphic electronics.

Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening

“…By summing these activities with the weights of corresponding synapses a neuron projects each data sample onto the vector of synaptic weights and transmits the projection to downstream neurons via its output activity. If synaptic weights are updated after each data sample presentation according to the Oja learning rule, a neuron computes the top eigenvector of the covariance matrix and outputs the first principle component [4], [5]. Here, we ignore temporal correlations in activity and assume that the dataset is presented as a sequence of static "snapshots" streamed in an arbitrary order.…”

Neuronal Morphology and Synapse Count in the Nematode Worm

“…One approach toward understanding synaptic function is from the theory of dynamical systems (Spruston, 2008; Hu et al, 2013). This includes knowledge for modeling the electrical and physiological properties of a neuronal cell and its reliability in communicating information across synaptic connections.…”

A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks

“…are optima of (3), where (5) uniquely defines all optima of (3), except when k < m, λ X k > αT and λ X k = λ X k+1 .…”

“…In the second phase of the algorithm, synaptic weights are updated for feedforward connections according to a local Hebbian rule (15) and for lateral connections according to a local anti-Hebbian rule (due to the (−) sign in equation ( 13)). Interestingly, in the α = 0 limit, these updates have the same form as the single-neuron Oja rule [24,2], except that the learning rate is not a free parameter but is determined by the cumulative neuronal activity 1/D Y T +1,i [4,5].…”

“…Following Oja's work, many multineuron circuits were proposed to extract multiple principal components of the input, for a review see [3]. However, most multineuron algorithms did not meet the same level of rigor and biological plausibility as the single-neuron algorithm [2,4] which can be derived using a normative approach, from a principled objective function [5], and contains only lo-cal Hebbian learning rules. Algorithms derived from principled objective functions either did not posess local learning rules [6,4,7,8] or had other biologically implausible features [9].…”