Deterministic networks for probabilistic computing

Jordan, Jakob; Petrovici, Mihai A.; Breitwieser, Oliver; Schemmel, Johannes; Meier, K.; Diesmann, Markus; Tetzlaff, Tom

doi:10.1038/s41598-019-54137-7

Cited by 16 publications

(15 citation statements)

References 74 publications

(164 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The additive noise ξ represents white noise with variance 2Cλ e , arising, for example, from unspecific background inputs [24,25]. For fixed presynaptic activity r, the average somatic membrane potential hence represents a maximum-a-posteriori estimate (MAP, [16]), while its variance is inversely proportional…”

Section: Bayesian Neuronal Dynamicsmentioning

confidence: 99%

Learning Bayes-optimal dendritic opinion pooling

Jordan¹,

Sacramento²,

Wybo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Pooling different opinions and weighting them according to their reliability is conducive to making good decisions. We demonstrate that single cortical neurons, through the biophysics of conductancebased coupling, perform such complex probabilistic computations via their natural dynamics. While the effective computation can be described as a feedforward process, the implementation critically relies on the bidirectional current flow along the dendritic tree. We suggest that dendritic membrane potentials and conductances encode opinions and their associated reliabilities, on which the soma acts as a decision maker. Furthermore, we derive gradient-based plasticity rules, allowing neurons to learn to represent desired target distributions and to weight afferents according to their reliability. Our theory shows how neurons perform Bayes-optimal cue integration. It also explains various experimental findings, both on the system and on the single-cell level, and makes new, testable predictions for intracortical neuron and synapse dynamics.

show abstract

Section: Bayesian Neuronal Dynamicsmentioning

confidence: 99%

Learning Bayes-optimal dendritic opinion pooling

Jordan¹,

Sacramento²,

Wybo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is because the response variability of cortical neurons observed in electrophysiological recordings has been well-explained in terms of probabilistic computation (Shadlen and Newsome, 1998). To date, stochastic computing algorithms based on restricted Boltzmann machine (Jordan et al, 2019) and Bayesian inference (Sountsov and Miller, 2015) have exhibited remarkable advantages in edge detection (Joe and Kim, 2019), traffic prediction (Sun X. et al, 2020), and the complex prediction of protein functions (Zou et al, 2017). However, the existing stochastic neural networks remain at quasistochastic states and are accelerated by the central processing unit or graphic processing unit.…”

Section: Introductionmentioning

confidence: 99%

Emerging Artificial Neuron Devices for Probabilistic Computing

Geng

Wang

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

In recent decades, artificial intelligence has been successively employed in the fields of finance, commerce, and other industries. However, imitating high-level brain functions, such as imagination and inference, pose several challenges as they are relevant to a particular type of noise in a biological neuron network. Probabilistic computing algorithms based on restricted Boltzmann machine and Bayesian inference that use silicon electronics have progressed significantly in terms of mimicking probabilistic inference. However, the quasi-random noise generated from additional circuits or algorithms presents a major challenge for silicon electronics to realize the true stochasticity of biological neuron systems. Artificial neurons based on emerging devices, such as memristors and ferroelectric field-effect transistors with inherent stochasticity can produce uncertain non-linear output spikes, which may be the key to make machine learning closer to the human brain. In this article, we present a comprehensive review of the recent advances in the emerging stochastic artificial neurons (SANs) in terms of probabilistic computing. We briefly introduce the biological neurons, neuron models, and silicon neurons before presenting the detailed working mechanisms of various SANs. Finally, the merits and demerits of silicon-based and emerging neurons are discussed, and the outlook for SANs is presented.

show abstract

“…It is straightforward to show that the CNOT-gate cannot be realized on the operator level. A realization on the operator level would require relations of the type (39) on the level of the classical operators B (µν) and A (µν) , which amounts to [S, A (30) ] = [S, A (01) ] = [S, A (31) ] = 0 , SA (11) = A (10) S , SA (10) = A (11) S , SA (21) = A (20) S , SA (20) = A (21) S , SA (33) = A (03) S , SA (03) = A (33) S , SA (32) = A (02) S , SA (02) = A (32) S , SA (22) = −A (13) S , SA (13) = −A (22) S , SA (12) = A (23) S , SA (23) = A (12) S .…”

Section: Cnot-gate In Probabilistic Computingmentioning

confidence: 99%

“…Since neural networks operate classically, this amounts to realizing quantum statistics by classical statistical systems. Neuromorphic computing [11][12][13][14][15][16][17], or computational steps in our brain, may also realize quantum operations based on classical statistics. Learning by evolution, living systems may be able to perform quantum computational steps without realizing conditions that are often assumed to be necessary for quantum mechanics, as the presence of small and well isolated systems or low temperature.…”

Section: Introductionmentioning

confidence: 99%

Quantum computing with classical bits

Wetterich

2019

Nuclear Physics B

View full text Add to dashboard Cite

A bit-quantum map relates probabilistic information for Ising spins or classical bits to quantum spins or qubits. Quantum systems are subsystems of classical statistical systems. The Ising spins can represent macroscopic two-level observables, and the quantum subsystem employs suitable expectation values and correlations. We discuss static memory materials based on Ising spins for which boundary information can be transported through the bulk in a generalized equilibrium state. They can realize quantum operations as the Hadamard or CNOT-gate for the quantum subsystem. Classical probabilistic systems can account for the entanglement of quantum spins. An arbitrary unitary evolution for an arbitrary number of quantum spins can be described by static memory materials for an infinite number of Ising spins which may, in turn, correspond to continuous variables or fields. We discuss discrete subsets of unitary operations realized by a finite number of Ising spins. They may be useful for new computational structures. We suggest that features of quantum computation or more general probabilistic computation may be realized by neural networks, neuromorphic computing or perhaps even the brain. This does neither require small isolated entities nor very low temperature. In these systems the processing of probabilistic information can be more general than for static memory materials. We propose a general formalism for probabilistic computing for which deterministic computing and quantum computing are special limiting cases.

show abstract

Deterministic networks for probabilistic computing

Cited by 16 publications

References 74 publications

Learning Bayes-optimal dendritic opinion pooling

Learning Bayes-optimal dendritic opinion pooling

Emerging Artificial Neuron Devices for Probabilistic Computing

Quantum computing with classical bits

Contact Info

Product

Resources

About