Probabilistic Models with Deep Neural Networks

Abstract

Recent advances in statistical inference have significantly expanded the toolbox of probabilistic modeling. Historically, probabilistic modeling has been constrained to very restricted model classes, where exact or approximate probabilistic inference is feasible. However, developments in variational inference, a general form of approximate probabilistic inference that originated in statistical physics, have enabled probabilistic modeling to overcome these limitations: (i) Approximate probabilistic inference is now possible over a broad class of probabilistic models containing a large number of parameters, and (ii) scalable inference methods based on stochastic gradient descent and distributed computing engines allow probabilistic modeling to be applied to massive data sets. One important practical consequence of these advances is the possibility to include deep neural networks within probabilistic models, thereby capturing complex non-linear stochastic relationships between the random variables. These advances, in conjunction with the release of novel probabilistic modeling toolboxes, have greatly expanded the scope of applications of probabilistic models, and allowed the models to take advantage of the recent strides made by the deep learning community. In this paper, we provide an overview of the main concepts, methods, and tools needed to use deep neural networks within a probabilistic modeling framework.

Exploring the Intersection of Artificial Intelligence and Clinical Healthcare: A Multidisciplinary Review

“…Network PNN A feedforward neural network used to handle classification and pattern recognition problems [43].…”

Performing Bayesian Network Inference Using Amortized Region Approximation with Graph Factorization

“…add an intermediate node Z with the CPD defned in ( 9) and reconnect all nodes; (6) convert p(X pa(j) | X pa(pa(j)) ) as hidden pairwise factors in 􏽢 G; (7) (15) for each unused edge e � (s, t) from a visited node s: //construct partial tree-robust cycles (16) if t is unvisited: (17) add an ear e 1 from s through edge e to some visited node u; (18) fnd the shortest path e 2 from s to u through used edges; (19) add cycle e 1 ∪ e 2 as outer region to G R ; (20) end if; (21) end for; (22) for each size > 3 outer region r(r ∈ R) in G R : (23) add chordal edges to the cycle corresponding to r in M( 􏽢 G) by following the node ordering π to make the cycle chordal; (24) replace r with its triplet decompositions r 1 , r 2 , . .…”

“…However, running VI or GBP is inefcient for large and noneconjugate exponential family models. Tis is because, under these circumstances, they feature an iterative VI [16] or GBP [11,17] message-passing procedure, in which computing gradients can be inconvenient [18] and sensitive to numerical issues [15].…”

“…However, unlike the VI-DNN, the BP-DNN does not ensure the estimated "none-ELBO" is of any bound to the ground truth but compares more favorably to the VI-DNN on accuracy [28]. Besides, the BP-DNN is more fexible than VI-DNN on parameter settings, because, for example, VI-DNN restricts the factorization of the latent variable distributions into simple forms, such as mean-feld factorization [9,18].…”

Can feature structure improve model’s precision? A novel prediction method using artificial image and image identification

“…Nonlinear models were adopted to solve those problems. 3 , 4 For example, nonlinear models represented by artificial neural networks have been introduced into epidemiological research. 5 , 6 …”