Meta-Analysis of Randomized Response Research

Abstract

This article discusses two meta-analyses on randomized response technique (RRT) studies, the first on 6 individual validation studies and the second on 32 comparative studies. The meta-analyses focus on the performance of RRTs compared to conventional question-and-answer methods. The authors use the percentage of incorrect answers as effect size for the individual validation studies and the standardized difference score (d-probit) as effect size for the comparative studies. Results indicate that compared to other methods, randomized response designs result in more valid data. For the individual validation studies, the mean percentage of incorrect answers for the RRT condition is .38; for the other conditions, it is .49. The more sensitive the topic under investigation, the higher the validity of RRT results. However, both meta-analyses have unexplained residual variances across studies, which indicates that RRTs are not completely under the control of the researcher.

Sensitive questions in surveys.

“…In the other common variation-the forced alternative method-the randomizing device determines whether the respondent is supposed to give a "yes" answer, a "no" answer, or an answer to the sensitive question. According to a metaanalysis of studies using RRT (Lensvelt-Mulders, Hox, van der Heijden, & Maas, 2005), the most frequently used randomizing devices are dice and coins. Table 8 below shows the formulas for deriving estimates from data obtained via the different RRT procedures and also gives the formulas for estimating the variance of those estimates.…”

“…The meta-analysis by Lensvelt-Mulders et al (2005) examined two types of studies that evaluated the effectiveness of RRT relative to other methods for collecting the same information, such as standard face-to-face interviewing. In one type, the researchers had validation data and could determine the respondent's actual status on the variable in question.…”

The logistic regression model with response variables subject to randomized response

“…For instance, it might be that some respondents do not trust the privacy protection offered by the RR design and give a socially desirable answer anyway. The general idea, however, is that RR performs relatively well, see Lensvelt-Mulders et al (2005). Future RR surveys may profit from research into cheating with respect to the RR design, see Böckenholt and van der Heijden (in press) and the references therein.…”

“…The misclassification protects the privacy of the individual respondent. A meta-analysis by Lensvelt-Mulders et al (2005) shows that RR yields more valid prevalence estimates than other methods for sensitive questions.…”

E-research – selected issues concerning the Internet attitudes and opinions research

“…This finding is consistent with previous reviews of the literature. Like Umesh and Peterson (1991), a recent review by Lensvelt-Mulders et al (2005b) concludes that "there have been very few substantive applications of RRTs [randomized response techniques] and that most papers are published to test a variant or illustrate a statistical problem" (p. 325).…”