Stratied Finite Empirical Bernstein Sampling

Abstract

We derive a concentration inequality for the uncertainty in the mean computed by stratified random sampling, and provide an online sampling method based on this inequality.  Our concentration inequality is versatile and considers a range of factors including: the data ranges, weights, sizes of the strata, the number of samples taken, the estimated sample variances, and whether strata are sampled with or without replacement.  Sequentially choosing samples to minimize this inequality leads to a online method for choosing samples from a stratified population.  We evaluate and compare the effectiveness of our method against others for synthetic data sets, and also in approximating the Shapley value of cooperative games.  Results show that our method is competitive with the performance of Neyman sampling with perfect variance information, even without having prior information on strata variances.We also provide a multidimensional extension of our inequality and discuss future applications.

Assessing Antithetic Sampling for Approximating Shapley, Banzhaf, and Owen Values

“…Our subject is limited to antithetic sampling with replacement. While our paper reports very favorable results for Neyman sampling, i.e., the two-stage stratified sampling algorithm from [22], we need to stress that we omitted other important stratification methods, in particular Bernstein sampling introduced by Burgess and Chapman in their papers [31,32]. We are certain it would not have changed our evaluation of antithetic sampling.…”

“…We are certain it would not have changed our evaluation of antithetic sampling. Also, we deliberately did not include sampling approaches without replacement [31,32] in this study. While these approaches allow for sharper error bounds, they are more sophisticated to implement and discuss as storage requirements might become more critical.…”

“…There are various other concepts for approximating Shapley values which this paper does not discuss, as it is too numerous to list them all. In particular, we do not perform any sampling without replacement [31,32] and we completely omit the multilinear extension method by Owen (1972) [33] and its recent variants [34], any approaches to combine sampling with exact solutions of subproblems [35,36], and any modern machine learning-related models [19,37].…”