No abstract

A combination of propensity score and calibration adjustment is shown to reduce bias in volunteer panel Web surveys. In this combination, the design weights are adjusted by propensity scores to correct for selection bias due to nonrandomized sampling. These adjusted weights are then calibrated to control totals for the target population and correct for coverage bias. The final set of weights is comprised of multiple components, and the estimator of a total no longer takes a linear form. Therefore, approximate methods are needed to derive variance estimates. This study compares three variance estimation methods through simulation. The first method resembles what is used in commercial statistical software based on squared residuals. The second approach uses a variance estimator originally derived for the generalized regression estimator. The third method uses jackknife replication. Results indicate bias reduction is crucial for valid variance estimation and favor the replication method over the other approaches.

No abstract

Panels of persons who volunteer to participate in Web surveys are used to make estimates for entire populations, including persons who have no access to the Internet. One method of adjusting a volunteer sample to attempt to make it representative of a larger population involves randomly selecting a reference sample from the larger population. The act of volunteering is treated as a quasi-random process where each person has some probability of volunteering. One option for computing weights for the volunteers is to combine the reference sample and Web volunteers and estimate probabilities of being a Web volunteer via propensity modeling. There are several options for using the estimated propensities to estimate population quantities. Careful analysis to justify these methods is lacking. The goals of this article are (a) to identify the assumptions and techniques of estimation that will lead to correct inference under the quasi-random approach, (b) to explore whether methods used in practice are biased, and (c) to illustrate the performance of some estimators that use estimated propensities. Two of our main findings are (a) that estimators of means based on estimates of propensity models that do not use the weights associated with the reference sample are biased even when the probability of volunteering is correctly modeled and (b) if the probability of volunteering is associated with analysis variables collected in the volunteer survey, propensity modeling does not correct bias.