Gibbs-type Indian Buffet Processes

Heaukulani, Creighton; Roy, Daniel M.

doi:10.1214/19-ba1166

Cited by 6 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Its widespread popularity is due to its power to generate a binary matrix with infinite columns. Restricted IBP [27] is proposed to allow an arbitrary prior distribution rather than a fixed Poisson distribution form for the number of features in each observation; Integrative IBP [28] is developed for integrating multimodal data in a latent space; Gibbs-type IBP [29] is a generalization of IBP with two-parameter IBP and three-parameter IBP [30] as special cases. All these models provide extensions for the original IBP to endow it with new meaningful features, e.g., power-law behavior.…”

Section: B Ibp and Dibpmentioning

confidence: 99%

Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet Processes

Xuan

Zhang

et al. 2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Sparse nonnegative matrix factorization (SNMF) aims to factorize a data matrix into two optimized nonnegative sparse factor matrices, which could benefit many tasks, such as document-word co-clustering. However, the traditional SNMF typically assumes the number of latent factors (i.e., dimensionality of the factor matrices) to be fixed. This assumption makes it inflexible in practice. In this paper, we propose a doubly sparse nonparametric NMF framework to mitigate this issue by using dependent Indian buffet processes (dIBP). We apply a correlation function for the generation of two stick weights associated with each column pair of factor matrices while still maintaining their respective marginal distribution specified by IBP. As a consequence, the generation of two factor matrices will be columnwise correlated. Under this framework, two classes of correlation function are proposed: 1) using bivariate Beta distribution and 2) using Copula function. Compared with the single IBP-based NMF, this paper jointly makes two factor matrices nonparametric and sparse, which could be applied to broader scenarios, such as co-clustering. This paper is seen to be much more flexible than Gaussian process-based and hierarchial Beta process-based dIBPs in terms of allowing the two corresponding binary matrix columns to have greater variations in their nonzero entries. Our experiments on synthetic data show the merits of this paper compared with the state-of-the-art models in respect of factorization efficiency, sparsity, and flexibility. Experiments on real-world data sets demonstrate the efficiency of this paper in document-word co-clustering tasks.

show abstract

Section: B Ibp and Dibpmentioning

confidence: 99%

Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet Processes

Xuan

Zhang

et al. 2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…The SB-IBP exhibits a number of power-law behaviors [Teh and Görür, 2009, Broderick et al, 2012, Heaukulani and Roy, 2018. First, the total number of non-zero columns of Z (in our case, observed vertices) in N rows (cliques) follows a power law, growing, as N → ∞, as…”

Section: Random Cliques Generated Using Exchangeable Random Measuresmentioning

confidence: 99%

“…This is a direct consequence of the power-law behavior of the SB-IBP: the number of vertices in a graph with N generating cliques correspond to the number of non-zero columns in a sample from the SB-IBP with N rows. Broderick et al [2012] and Heaukulani and Roy [2018] show that the expected number of non-zero columns grows as α σ…”

Section: Sparsitymentioning

confidence: 99%

Random clique covers for graphs with local density and global sparsity

Williamson,

Tec

2018

Preprint

View full text Add to dashboard Cite

Large real-world graphs tend to be sparse, but they often contain many densely connected subgraphs and exhibit high clustering coefficients. While recent random graph models can capture this sparsity, they ignore the local density, or vice versa. We develop a Bayesian nonparametric graph model based on random edge clique covers, and show that this model can capture power law degree distribution, global sparsity and non-vanishing local clustering coefficient. This distribution can be used directly as a prior on observed graphs, or as part of a hierarchical Bayesian model for inferring latent graph structures.

show abstract

“…The generalized CUSP prior subsumes several specific priors involving stickbreaking representations from beta distributions that were introduced earlier in the literature for factor-analytical models, see e.g. [9,15,[19][20][21].…”

Section: Introductionmentioning

confidence: 99%

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis

Frühwirth‐Schnatter

2023

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

The paper discusses shrinkage priors which impose increasing shrinkage in a sequence of parameters. We review the cumulative shrinkage process (CUSP) prior of Legramanti et al. (Legramanti et al . 2020 Biometrika 107 , 745–752. ( doi:10.1093/biomet/asaa008 )), which is a spike-and-slab shrinkage prior where the spike probability is stochastically increasing and constructed from the stick-breaking representation of a Dirichlet process prior. As a first contribution, this CUSP prior is extended by involving arbitrary stick-breaking representations arising from beta distributions. As a second contribution, we prove that exchangeable spike-and-slab priors, which are popular and widely used in sparse Bayesian factor analysis, can be represented as a finite generalized CUSP prior, which is easily obtained from the decreasing order statistics of the slab probabilities. Hence, exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index in the loading matrix increases, without imposing explicit order constraints on the slab probabilities. An application to sparse Bayesian factor analysis illustrates the usefulness of the findings of this paper. A new exchangeable spike-and-slab shrinkage prior based on the triple gamma prior of Cadonna et al. (Cadonna et al . 2020 Econometrics 8 , 20. ( doi:10.3390/econometrics8020020 )) is introduced and shown to be helpful for estimating the unknown number of factors in a simulation study. This article is part of the theme issue ‘Bayesian inference: challenges, perspectives, and prospects’.

show abstract

Gibbs-type Indian Buffet Processes

Cited by 6 publications

References 34 publications

Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet Processes

Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet Processes

Random clique covers for graphs with local density and global sparsity

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis

Contact Info

Product

Resources

About