A flexible regression model for count data

Abstract

Poisson regression is a popular tool for modeling count data and is applied in a vast array of applications from the social to the physical sciences and beyond. Real data, however, are often over- or under-dispersed and, thus, not conducive to Poisson regression. We propose a regression model based on the Conway--Maxwell-Poisson (COM-Poisson) distribution to address this problem. The COM-Poisson regression generalizes the well-known Poisson and logistic regression models, and is suitable for fitting count data with a wide range of dispersion levels. With a GLM approach that takes advantage of exponential family properties, we discuss model estimation, inference, diagnostics, and interpretation, and present a test for determining the need for a COM-Poisson regression over a standard Poisson regression. We compare the COM-Poisson to several alternatives and illustrate its advantages and usefulness using three data sets with varying dispersion.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS306 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Generalised count distributions for modelling parity

“…Thirdly, the purpose of modelling may be decisive, because the two distributions differ in their practical properties. While the statistical properties of the COM-Poisson model have been investigated more fully (Sellers and Shmueli 2010) and asymptotic significance tests are available, the gamma count model has the advantage that it appears to be vastly more efficient computationally, by one or two orders of magnitude. This is no doubt due to the fact that the underlying gamma function benefits from being a common mathematical function for which highly optimised algorithms are standard.…”

Accounting for Dispersion and Correlation in Estimating Safety Performance Functions. An Overview Starting from a Case Study

“…Moreover, the Conway-Maxwell-Poisson (COM-Poisson) distribution has been re-introduced by statisticians to model count data characterized by either over-or under-dispersion (Shmueli et al, 2005;Guikema & Coffelt, 2008;Zou et al, 2011). The COM-Poisson distribution was first introduced in 1962 by Conway and Maxwell; only in 2008 it was evaluated in the context of a GLM by Guikema and Coffelt (2008), Lord et al (2008) and Sellers and Shmueli (2010). The COM-Poisson distribution is a two parameter generalization of the Poisson distribution that is flexible enough to describe a wide range of count data distributions (Sellers & Shmueli, 2010); since its revival, it has been further developed in several directions and applied in multiple fields (Sellers et al, 2011).…”

“… tj = a centering parameter, denoting the expected value under a Poisson distribution associated with the generic observation at year t and at site j (Sellers and Shmueli, 2010);…”

“…Once the dispersion parameter was estimated its value was used to obtain new estimates of model parameters; this iteration was performed until all the values (i.e., dispersion parameter, coefficients) converged. COM-Poisson model was estimated using R software, after implementation of codes arranged by Sellers and Shmueli (2010), available at www9.georgetown.edu/faculty/kfs7/research. Table 4 shows coefficients estimates, standard errors and goodness-of-fit statistics for the Poisson model (model 1), the quasi-Poisson model (model 2), the COM-Poisson model (model 3).…”

“…Table 4 shows coefficients estimates, standard errors and goodness-of-fit statistics for the Poisson model (model 1), the quasi-Poisson model (model 2), the COM-Poisson model (model 3). COM-Poisson coefficients are for centering parameter  (see equation 6) and not for mean as in the case of Poisson/quasi-Poisson model Sellers & Shmueli 2010); this is why COM-Poisson coefficients cannot be directly compared with the Poisson/quasi-Poisson ones. From results showed in Table 4, although there is no difference between parameters estimates (and GOF) for Poisson and quasi-Poisson models, it can be seen that the consideration of underdispersion in the data improves the estimates accuracy, as it is shown by the reductions in the standard errors values.…”