Optimal Stratification and Allocation for the June Agricultural Survey

Abstract

A computational approach to optimal multivariate designs with respect to stratification and allocation is investigated under the assumptions of fixed total allocation, known number of strata, and the availability of administrative data correlated with thevariables of interest under coefficient-of-variation constraints. This approach uses a penalized objective function that is optimized by simulated annealing through exchanging sampling units and sample allocations among strata. Computational speed is improved through the use of a computationally efficient machine learning method such as K-means to create an initial stratification close to the optimal stratification. The numeric stability of the algorithm has been investigated and parallel processing has been employed where appropriate. Results are presented for both simulated data and USDA's June Agricultural Survey. An R package has also been made available for evaluation.

Heuristic approach applied to the optimum stratification problem

“…Most of the proposed methods in the literature are related to the objective function described in (i), as can be seen in [2,3,5,6,[10][11][12][13][14][15][16]18,22,25,26,[37][38][39][40]. The objective function described in (ii) was only studied by Hidiroglou [23], Lavallée and Hidiroglou [33], Kozak [28], Hidiroglou and Kozak [24] and Lisic et al [35]. Note that both objective functions are correlated since minimizing variance requires the sample size as an input, and minimizing the sample size requires precision (that is, variance or variation coefficient) as an input to the problem.…”

“…The optimum stratification problem, related to the field of probability sampling [8], can be formulated according to two possible goals: (A) minimizing the variance of an estimator given a fixed sample size or (B) minimizing the sample size for a fixed level of precision. In the literature, most methods were developed aiming at the first goal [2,3,5,6,10,15,18,22,25,26,30,[37][38][39], while the second goal has been less studied [23,24,28,33,35]. This article's optimization problem consists of minimizing the total sample size, simultaneously satisfying the constraints of precision and minimum sample size of each stratum.…”

Combining K-means type algorithms with Hill Climbing for Joint Stratification and Sample Allocation Designs

“…The equivalency of the stratification aspect of the joint stratification and sample allocation problem to a machine learning clustering technique, the K-means method, was first noted by (Lisic et al, 2018). This refers to a family of algorithms "developed as a result of independent investigations in the 1950s" (Pérez-Ortega et al, 2019) but first given the name K-means by (MacQueen et al, 1967).…”

“…Following (Lisic et al, 2018)'s proposal (Ballin and Barcaroli, 2020) and (O'Luing et al, 2020) used an efficient version of the K-means clustering algorithm (Hartigan, 1979) to determine initial solutions for our problem which are then improved upon or "optimised" using a grouping genetic algorithm and simulated annealing algorithm respectively. (Hartigan, 1979)'s algorithm searches for a solution that is locally optimal and although it is quick, alternative solutions to that found by the algorithm may have the same or smaller within sums of squares.…”

A Simulated Annealing Algorithm for Joint Stratification and Sample Allocation Designs

“…Once they are created we obtain the mean and standard deviation of the relevant observation values from the one or more target variable columns that fall within each atomic stratum. (Lisic et al, 2018) refers to atomic strata as primary sampling units (PSUs). This more clearly distinguishes them from the strata that they are subsequently partitioned into, however, we will continue to use the term atomic strata as we wish to build on the progress made by (Benedetti et al, 2008;Ballin and Barcaroli, 2013) and (O'Luing et al, 2019) on developing algorithms to solve this problem.…”