Neural network implementation of inference on binary Markov random fields with probability coding

Yu, Zhaofei; Chen, Feng; Dong, Jianwu

doi:10.1016/j.amc.2016.12.025

Cited by 7 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to the studies in [26,27], we assume that J ij (x i , x j ) = J ij x i x j and observations. In this paper, we only consider marginal inference.…”

Section: Markov Random Fields and Marginal Inferencementioning

confidence: 99%

“…Similar to the studies in [26,27], we only consider inference of binary MRFs in this paper, which means the value of the variable x i can be 1 or -1 (x i = 1 or −1).…”

Section: Converting Mean-field Inference Into a Differential Equationmentioning

confidence: 99%

“…In order to propose biologically more plausible neural networks to implement inference, Ott and Stoop [26] Yu et al [27] went a further step to relax the constraints on initialized messages, but still required the special topological structure and potential function of MRFs. Another important way is based on tree-based reparameterization algorithm [28], which, however, is only limited to the case of exponential family distributions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Zheng

Jia

et al. 2020

Neural Networks

Self Cite

View full text Add to dashboard Cite

Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in the brain. Previous studies have tried to relate the inference equation of binary Markov random fields to the dynamic equation of spiking neural networks through belief propagation algorithm and reparameterization, but they are valid only for Markov random fields with limited network structure. In this paper, we propose a spiking neural network model that can implement inference of arbitrary binary Markov random fields. Specifically, we design a spiking recurrent neural network and prove that its neuronal dynamics are mathematically equivalent to the inference process of Markov random fields by adopting mean-field theory. Furthermore, our mean-field approach unifies previous works.Theoretical analysis and experimental results, together with the application to image denoising, demonstrate that our proposed spiking neural network can get comparable results to that of mean-field inference.

show abstract

“…Similar to the studies in [26,27], we assume that J ij (x i , x j ) = J ij x i x j and observations. In this paper, we only consider marginal inference.…”

Section: Markov Random Fields and Marginal Inferencementioning

confidence: 99%

“…Similar to the studies in [26,27], we only consider inference of binary MRFs in this paper, which means the value of the variable x i can be 1 or -1 (x i = 1 or −1).…”

Section: Converting Mean-field Inference Into a Differential Equationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Zheng

Jia

et al. 2020

Neural Networks

Self Cite

View full text Add to dashboard Cite

show abstract

“…Beck and Pouget [7] went a future step to solve the approximation problem and came up with a precise equivalence relation. Similarly, Ott et al [8] and Yu et al [9] built the relationship between inference equation of Markov random fields and the dynamics of recurrent neural networks with BP. The above works based on equivalence proof are only appropriate for small-scale Bayesian inference.…”

Section: Introductionmentioning

confidence: 99%

Implementation of Bayesian Inference In Distributed Neural Networks

Huang

Liu

2018

2018 26th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

Self Cite

View full text Add to dashboard Cite

Numerous neuroscience experiments have suggested that the cognitive process of human brain is realized as probability reasoning and further modeled as Bayesian inference. It is still unclear how Bayesian inference could be implemented by neural underpinnings in the brain. Here we present a novel Bayesian inference algorithm based on importance sampling. By distributed sampling through a deep tree structure with simple and stackable basic motifs for any given neural circuit, one can perform local inference while guaranteeing the accuracy of global inference. We show that these task-independent motifs can be used in parallel for fast inference without iteration and scale-limitation. Furthermore, experimental simulations with a small-scale neural network demonstrate that our distributed sampling-based algorithm, consisting with our theoretical analysis, can approximate Bayesian inference. Taken all together, we provide a proofof-principle to use distributed neural networks to implement Bayesian inference, which gives a road-map for large-scale Bayesian network implementation based on spiking neural networks with computer hardwares, including neuromorphic chips.

show abstract

“…In this way, the summation of probabilities can be calculated by summing the overall responses of neurons. The same way of coding was also used in [27]. In order to simplify the multiplication of probabilities, Rao [28], [29] proposed to use log probability code and proved that the differential equations of recurrent neural networks are in coincidence with the inference equations of hidden Markov model, in which the computation of sum-logs was used to approximate the computation of log-sum.…”

mentioning

confidence: 99%

Sampling-Tree Model: Efficient Implementation of Distributed Bayesian Inference in Neural Networks

Chen

Liu

2020

IEEE Trans. Cogn. Dev. Syst.

Self Cite

View full text Add to dashboard Cite

Experimental observations from neuroscience have suggested that the cognitive process of human brain is realized as probabilistic reasoning and further modelled as Bayesian inference. However, it remains unclear how Bayesian inference could be implemented by network of neurons in the brain. Here a novel implementation of neural circuit, named sampling-tree model, is proposed to fulfill this aim. By using a deep tree structure to implement sampling with simple and stackable basic neural network motifs for any given Bayesian networks, one can perform local inference while guaranteeing the accuracy of global inference. We show that these task-independent motifs can be used in parallel for fast inference without intensive iteration and scale-limitation. As a result, this model utilizes the structure benefit of neuronal system, i.e., neuronal abundance and multihierarchy, to perform fast inference in an extendable way. Index Terms-Sampling-tree model, neural network, Bayesian inference, importance sampling, probabilistic population coding I. INTRODUCTION U NDERSTANDING how the brain works is one of the most challenging problems in 21 century. Our brain can represent probability distribution [1], [2], [3]. The cognitive and perceptive process of the brain is a process of probabilistic reasoning, which has been indicated by a number of psychological and neuroscience experiments [4], [5]. From the macroscopic level, Bayesian models have shown their ability of explaining how the brain perceives the world and have been successfully used in various fields of brain

show abstract

Neural network implementation of inference on binary Markov random fields with probability coding

Cited by 7 publications

References 15 publications

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Implementation of Bayesian Inference In Distributed Neural Networks

Sampling-Tree Model: Efficient Implementation of Distributed Bayesian Inference in Neural Networks

Contact Info

Product

Resources

About