Estimating the size of a hidden finite set: Large-sample behavior of estimators

Abstract

A finite set is "hidden" if its elements are not directly enumerable or if its size cannot be ascertained via a deterministic query. In public health, epidemiology, demography, ecology and intelligence analysis, researchers have developed a wide variety of indirect statistical approaches, under different models for sampling and observation, for estimating the size of a hidden set. Some methods make use of random sampling with known or estimable sampling probabilities, and others make structural assumptions about relationships (e.g. ordering or network information) between the elements that comprise the hidden set. In this review, we describe models and methods for learning about the size of a hidden finite set, with special attention to asymptotic properties of estimators. We study the properties of these methods under two asymptotic regimes, "infill" in which the number of fixed-size samples increases, but the population size remains constant, and "outfill" in which the sample size and population size grow together. Statistical properties under these two regimes can be dramatically different.

A Bayesian Treatment of the German Tank Problem

“…The German tank problem in other contexts. The Bayesian probability theory used to solve the German tank problem applies (perhaps with modification) to many other contexts in which we wish to estimate the size of some finite hidden set [ 9 ], such as the number of taxicabs in a city [ 19 , 23 ], racing cars on a track [ 48 ], accounts at a bank [ 25 ], pieces of furniture purchased by a university [ 22 ], aircraft operations at an airport [ 34 ], cases in court [ 50 ], or electronic devices produced by a company [ 2 ]. And also the extent of leaked classified government communications [ 18 ], the time needed to complete a project deadline [ 16 ], the time-coverage of historical records of extreme events like floods [ 39 ], the length of a short-tandem repeat allele [ 51 ], the size of a social network [ 28 ], the lifetime of a flower of a plant [ 38 ], or the duration of existence of a species [ 42 ].…”

Comparing the Robustness of Simple Network Scale-Up Method Estimators

“…To investigate this, we assume a model for link formation in the general population, then consider the distribution of the estimators over simple random samples from that population without replacement. The binomial model can be viewed as an approximation to the Erdős-Rényi network model, in which the presence or absence of a link between each pair of nodes is independently drawn from the same Bernoulli ( p ) distribution (for details, see the online supplement and Cheng, Eck, and Crawford 2020). In this study, we generalize this model to incorporate barrier effects by using a stochastic blockmodel (SBM) that partitions the general population into two mutually exclusive groups, the hard-to-reach group of interest ( H ) and everyone else ( L ) .…”

A Bayesian treatment of the German tank problem

“…The German tank problem in other contexts. The Bayesian probability theory to solve the German tank problem applies (perhaps, with modification) to many other contexts where we wish to estimate the size of some finite, hidden set [27], eg. : the number of taxicabs in a city [12], the number of accounts at a bank [15], the number of furniture pieces purchased by a university [10], the number of aircraft operations at an airport [28], the extent of leaked classified government communications [29], the time needed to complete a project deadline [30], the time-coverage of historical records of extreme events like floods [31], the length of a short-tandem repeat allele [32], the size of a social network [33], the number of cases in court [34], the lifetime of a flower of a plant [35], or the duration of existence of a species [36].…”