Priors for Random Count Matrices Derived from a Family of Negative Binomial Processes

Abstract

We define a family of probability distributions for random count matrices with a potentially unbounded number of rows and columns. The three distributions we consider are derived from the gamma-Poisson, gamma-negative binomial, and beta-negative binomial processes, which we refer to generically as a family of negative-binomial processes. Because the models lead to closed-form update equations within the context of a Gibbs sampler, they are natural candidates for nonparametric Bayesian priors over count matrices. A key aspect of our analysis is the recognition that, although the random count matrices within the family are defined by a rowwise construction, their columns can be shown to be independent and identically distributed. This fact is used to derive explicit formulas for drawing all the columns at once. Moreover, by analyzing these matrices' combinatorial structure, we describe how to sequentially construct a column-i.i.d. random count matrix one row at a time, and derive the predictive distribution of a new row count vector with previously unseen features. We describe the similarities and differences between the three priors, and argue that the greater flexibility of the gamma-and beta-negative binomial processes-especially their ability to model over-dispersed, heavytailed count data-makes these well suited to a wide variety of real-world applications. As an example of our framework, we construct a naive-Bayes text classifier to categorize a count vector to one of several existing random count matrices of different categories. The classifier supports an unbounded number of features, and unlike most existing methods, it does not require a predefined finite vocabulary to be shared by all the categories, and needs neither feature selection nor parameter tuning. Both the gamma-and beta-negative binomial processes are shown to significantly outperform the gamma-Poisson process when applied to document categorization, with comparable performance to other state-of-the-art supervised text classification algorithms.The authors are with the

BNP-Seq: Bayesian Nonparametric Differential Expression Analysis of Sequencing Count Data

“…Note that if we fix q j = 1 for all j, then the proposed NBP with sample-specific scaling parameters reduces to the NBP in Zhou and Carin [2015] and Zhou et al [2016].…”

“…possible ways to the columns of N J . Similar to the derivation in Zhou et al [2016], using a marginalization procedure shown in Caron et al [2014], one may marginalize out the gamma process G, leading to the distribution of the random count matrix as…”

“…Instead of using the scaled NBP that introduces q j to model sample-specific sequencing depths, we also consider the original NBP of Zhou et al [2016] with all q j fixed at one.…”

“…More specifically, moving beyond existing algorithms that model over-dispersed counts with the NB distribution, our Bayesian nonparametric (BNP) algorithms model the gene counts using the gamma-negative binomial process (GNBP) [Zhou et al, 2016], which mixes the NB shape parameter for each gene with the distribution of the weight of an atom of a gamma process [Ferguson, 1973], or beta-negative binomial process (BNBP) [Zhou et al, 2012, Zhou and Carin, 2015, Broderick et al, 2015, which mixes the NB probability parameter of each gene with the distribution of the weight of an atom of a beta process [Hjort, 1990]. In addition to the GNBP and BNBP, for comparison, we have extended the negative binomial process (NBP) of Zhou et al [2016] by multiplying the gene-specific Poisson rates with gamma distributed sample-specific scaling parameters, and refer to it as the scaled NBP. While the NBP of Zhou et al [2016] is not expected to work well since it does not explicitly model the variation of a sample's total count, the scaled NBP, even with a scaling parameter for each sample to capture that variation, is found to provide poor performance, indicating a clear limitation of the Poisson distribution assumption.…”

“…In addition to the GNBP and BNBP, for comparison, we have extended the negative binomial process (NBP) of Zhou et al [2016] by multiplying the gene-specific Poisson rates with gamma distributed sample-specific scaling parameters, and refer to it as the scaled NBP. While the NBP of Zhou et al [2016] is not expected to work well since it does not explicitly model the variation of a sample's total count, the scaled NBP, even with a scaling parameter for each sample to capture that variation, is found to provide poor performance, indicating a clear limitation of the Poisson distribution assumption.…”