Ensemble confidence intervals for binomial proportions

Abstract

We propose two measures of performance for a confidence interval for a binomial proportion p: the root mean squared error and the mean absolute deviation. We also devise a confidence interval for p based on the actual coverage function that combines several existing approximate confidence intervals. This "Ensemble" confidence interval has improved statistical properties over the constituent confidence intervals. Software in an R package, which can be used in devising and assessing these confidence intervals, is available on CRAN.

Supplemental Material: Precision and accuracy of modal analysis methods for clastic deposits and rocks: A statistical and numerical modeling approach

“…The Z-score for a 68.27% confidence level is of course 1.00 and that for a 95.45% confidence level is 2.00. This traditional method of using the normal distribution to find confidence intervals for a binomial proportion is called a "Wald interval" in the statistics literature (e.g., Vollset, 1993;Brown et al, 2002;Park and Leemis, 2019).…”

“…the lower bounds and upper bounds are not equal (Howarth, 1998), except when p = 0.5. There are many different ways to obtain lower and upper confidence bounds for the binomial distribution, as discussed for example by Vollset (1993), Howarth (1998) or Park and Leemis (2019). We show three of them for a proportion of 0.1 and confidence levels of 68.27% and 95.45% in figure S1b.…”

“…In both cases, the confidence bounds are asymmetrical. The Clopper-Pearson method aims to provide coverage equal to or higher than the specified confidence level (Park and Leemis, 2019). In other words, it is a conservative method that may overestimate the width of the confidence bounds to remain "on the safe side".…”

“…Although this method was recommended by Howarth (1998), we do not use it further in this article. Finally, the Wilsonscore method is recommended by Park and Leemis (2019) when N is 49 or more and the goal is to "minimize the absolute difference between the stated and actual coverage", which is what we aim for here. At very large N, all of these methods converge.…”

Binomial confidence intervals for rare events: importance of defining margin of error relative to magnitude of proportion

“…Several techniques have been devised to estimate a binomial proportion, p, including the Wald, Clopper-Pearson, Wilson, Agresti-Coull, Jeffreys, arcsine transformation, Jeffreys' Prior and the likelihood ratio interval. A range of ensemble/model-averaged approaches have also been considered, e.g., Turek and Fletcher (2012), Kabaila et al (2016) and Park and Leemis (2019). In this work we assess the performance of the Wald, Clopper-Pearson, Wilson and Agresti-Coull intervals in standard form, i.e., without application of a modification or continuity correction.…”