A Survey of Non-Exchangeable Priors for Bayesian Nonparametric Models

Foti, Nicholas J.; Williamson, Sinead A.

doi:10.1109/tpami.2013.224

Cited by 35 publications

(29 citation statements)

References 50 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The direct evaluation of (26) is unfeasible and one needs to resort to some simulation scheme. To this end, one may rely on the pEPPF in (12)- (15) to devise a Blackwell-MacQueen urn scheme, for any d ≥ 2, that generates X (mi|Ni) for any hierarchical NRMI. In order to simplify the notation and the description of the algorithm, we consider the case d = 2.…”

Section: Blackwell-macqueen Urn Schemementioning

confidence: 99%

See 1 more Smart Citation

Distribution theory for hierarchical processes

Camerlenghi¹,

Lijoi²,

Orbanz³

et al. 2019

Ann. Statist.

View full text Add to dashboard Cite

Hierarchies of discrete probability measures are remarkably popular as nonparametric priors in applications, arguably due to two key properties: (i) they naturally represent multiple heterogeneous populations; (ii) they produce ties across populations, resulting in a shrinkage property often described as "sharing of information". In this paper we establish a distribution theory for hierarchical random measures that are generated via normalization, thus encompassing both the hierarchical Dirichlet and hierarchical Pitman-Yor processes. These results provide a probabilistic characterization of the induced (partially exchangeable) partition structure, including the distribution and the asymptotics of the number of partition sets, and a complete posterior characterization. They are obtained by representing hierarchical processes in terms of completely random measures, and by applying a novel technique for deriving the associated distributions. Moreover, they also serve as building blocks for new simulation algorithms, and we derive marginal and conditional algorithms for Bayesian inference.

show abstract

Section: Blackwell-macqueen Urn Schemementioning

confidence: 99%

“…Such characterizations are of theoretical interest, but also a prerequisite for inference algorithms, which simulate draws from (unobserved) random measures conditionally on data. See [5,18,31,46] for examples, and [15] for a comprehensive list of references.…”

Section: Introductionmentioning

confidence: 99%

Distribution theory for hierarchical processes

Camerlenghi¹,

Lijoi²,

Orbanz³

et al. 2019

Ann. Statist.

View full text Add to dashboard Cite

show abstract

“…The Dirichlet process (DP) [32], [33] is a stochastic process used for Bayesian nonparametric data analysis, particularly in a DP mixture model (infinite mixture model). It is a distribution over distributions rather than parameters, i.e., each draw from a DP is a probability distribution itself, rather than a parameter vector [47].…”

Section: B Dirichlet Process With Stick-breakingmentioning

confidence: 99%

“…In other words, this approach allows the number of components to increase as new data arrives, which is the key difference from finite mixture modeling. The most widely used Bayesian nonparametric [31] model selection method is based on the Dirichlet process (DP) mixture model [32], [33]. The DP mixture model extends distributions over measures, which has the appealing property that it does not need to set a prior on the number of components.…”

mentioning

confidence: 99%

Variational Bayesian Learning for Dirichlet Process Mixture of Inverted Dirichlet Distributions in Non-Gaussian Image Feature Modeling

Lai

Kleijn

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

In this paper, we develop a novel variational Bayesian learning method for the Dirichlet process (DP) mixture of the inverted Dirichlet distributions, which has been shown to be very flexible for modeling vectors with positive elements. The recently proposed extended variational inference (EVI) framework is adopted to derive an analytically tractable solution. The convergency of the proposed algorithm is theoretically guaranteed by introducing single lower bound approximation to the original objective function in the EVI framework. In principle, the proposed model can be viewed as an infinite inverted Dirichlet mixture model that allows the automatic determination of the number of mixture components from data. Therefore, the problem of predetermining the optimal number of mixing components has been overcome. Moreover, the problems of overfitting and underfitting are avoided by the Bayesian estimation approach. Compared with several recently proposed DP-related methods and conventional applied methods, the good performance and effectiveness of the proposed method have been demonstrated with both synthesized data and real data evaluations.

show abstract

“…Finally, there has been a lot of research in Bayesian nonparametrics about dependent random measures, originating from the work of MacEachern (1999), broadly surveyed in Foti et al (2015), and used in applications such as for dynamic ordinal data (DeYoreo and Kottas, 2015), neuron spikes (Gasthaus et al, 2009), and images (Sudderth and Jordan, 2009). Dependent random measures select atoms for each observation through a priori covariates, such as a timestamp associated with the observation.…”

Section: Introductionmentioning

confidence: 99%

Correlated Random Measures

Ranganath

Blei

2017

Journal of the American Statistical Association

View full text Add to dashboard Cite

We develop correlated random measures, random measures where the atom weights can exhibit a flexible pattern of dependence, and use them to develop powerful hierarchical Bayesian nonparametric models. Hierarchical Bayesian nonparametric models are usually built from completely random measures, a Poisson-process based construction in which the atom weights are independent. Completely random measures imply strong independence assumptions in the corresponding hierarchical model, and these assumptions are often misplaced in real-world settings. Correlated random measures address this limitation. They model correlation within the measure by using a Gaussian process in concert with the Poisson process. With correlated random measures, for example, we can develop a latent feature model for which we can infer both the properties of the latent features and their dependency pattern. We develop several other examples as well. We study a correlated random measure model of pairwise count data. We derive an efficient variational inference algorithm and show improved predictive performance on large data sets of documents, web clicks, and electronic health records.

show abstract

A Survey of Non-Exchangeable Priors for Bayesian Nonparametric Models

Cited by 35 publications

References 50 publications

Distribution theory for hierarchical processes

Distribution theory for hierarchical processes

Variational Bayesian Learning for Dirichlet Process Mixture of Inverted Dirichlet Distributions in Non-Gaussian Image Feature Modeling

Correlated Random Measures

Contact Info

Product

Resources

About