The Effectiveness of Increasing Sample Size to Mitigate the Influence of Population Characteristics in Haphazard Sampling

Abstract

Over 40 years ago both Deming (1954) and Arkin (1957) expressed concerns that the composition of samples chosen through haphazard selection may be unrepresentative due to the presence of unintended selection biases. To mitigate this problem some experts in the field of audit sampling recommend increasing sample sizes by up to 100 percent when utilizing haphazard selection. To examine the effectiveness of this recommendation 142 participants selected haphazard samples from two populations. The compositions of these samples were then analyzed to determine if certain population elements were overrepresented, and if the extent of overrepresentation declined as sample size increased. Analyses disclosed that certain population elements were overrepresented in the samples. Also, increasing sample size produced no statistically significant change in the composition of samples from one population, while in the second population increasing the sample size produced a statistically significant but minor reduction in overrepresentation. These results suggest that individuals may be incapable of complying with audit guidelines that haphazard sample selections be made without regard to the observable physical features of population elements and cast doubt on the effectiveness of using larger sample sizes to mitigate the problem. Given these findings, standard-setting bodies should reconsider the conditions under which haphazard sampling is sanctioned as a reliable audit tool.

Sampling Practices of Auditors in Public Accounting, Industry, and Government (Retracted)

“…Identical questions about training in nonstatistical techniques appeared in section six of the instrument. In addition, because several published studies document selection biases (see, for example, Hall et al 2001;Yates 1937) and evaluation biases (see, for example, Ponemon and Wendell 1995;Kinney andUecker 1977a, 1977b) inherent in nonstatistical methods, the survey instrument asked respondents to report on the extent of training received in bias avoidance.…”

“…In section seven, respondents indicated on a seven-point scale (1 = none, 4 = some, 7 = great) the extent to which certain physical characteristics of population elements-color, size, and physical location-influence auditors' nonstatistical selections. The survey instrument identified these particular physical characteristics because published research indicates they do influence nonstatistical sample selections (see Hall et al 2001;Hall et al 2000) and because AICPA literature states that these characteristics should not influence sample selections (AICPA 1995, §8220.05). Section eight listed four approaches to deal with nonstatistical selection biases: (1) do nothing, (2) increase sample size, (3) stratify the sample, and (4) other (fill in the blank), and asked respondents to indicate which approach they use in practice.…”

Haphazard Sampling: Selection Biases and the Estimation Consequences of These Biases

“…The combined findings of Hall et al (2012) and those of Hall et al (2000Hall et al ( , 2001 suggest that the properties of haphazard samples, whether chosen from control listings or from the actual population, are likely to differ from those of random samples. Subconscious effort minimization and diversification behaviors, coupled with visual perception artifacts, yield samples that violate requirements for independence and equal selection probability.…”

“…However, a number of sampling experts have expressed doubts that haphazard sampling is a reliable substitute for random sampling (Deming 1954;Arkin 1957;Wilburn 1984). To investigate the validity of these concerns, Hall et al (2000Hall et al ( , 2001 tested haphazard samples chosen directly from populations and found evidence of unequal selection likelihoods. Our study extends this line of research by testing whether the properties of haphazard samples chosen from control listings exhibit the essential properties of random samples (i.e., independence and equal probability of selection).…”

“…When auditors use nonstatistical techniques, they should undertake and document debiasing efforts. One debiasing procedure in current use, increasing sample size to reduce haphazard sampling selection bias, has been shown to provide a small reduction in selection bias (Hall et al 2001). Another debiasing strategy is to avoid both: (1) always starting the selection process on the first page of control listings, and (2) proceeding through control listings in a serial fashion.…”