A Method to Obtain a Randomized Control Group where it Seems Impossible

Abstract

Randomization of program participants into control and experimental groups is often not feasible in field settings. The researcher's desire to evaluate a program with a rigorous experimental design is often incompatible with the objective of serving the expressed needs of the program participants. However, opportunities do arise when a randomized control group can be constructed without disregarding the participants' wish to be included in the treatment group. This article describes a method that uses the participants' compliance to program instructions as a means of classifying participants and, thereby, obtains a randomized control group for a subset of participants. A large smoking intervention project is used to illustrate two variations of this method.

Evidence of Real-World Effectiveness of a Telephone Quitline for Smokers

“…However, we found a significant difference in abstinence rates between the overall treatment and control groups, a comparison that was not dependent on any assumptions about the equality of treatment subgroup A and control subgroup A. 10,11 Although this study shows that the quitline's counseling was effective, its direct clinical effect is limited by the fact that most smokers in the program had relapsed by the end of 12 months. Nevertheless, the doubling effect of counseling on success rates compares favorably with the clinical-trial results summarized by the Public Health Service in its recommendation for telephone counseling.…”

“…The quitting rate in treatment subgroup B (Q TB ) was estimated by the following formula: Q TB =(Q T ¡w CA Q CA )÷w CB , where Q T is the quitting rate in the treatment group, Q CA is the quitting rate in control subgroup A, and w CA and w CB are the relative proportions of participants in control subgroup A and control subgroup B, respectively (w CA + w CB =1). 10 Multiple-outcome measures were used in the analysis, according to the recommendations of the Society for Research on Nicotine and Tobacco for assessing treatment effects. 15 The primary measure of the overall intervention effect was prolonged abstinence on the basis of an intention-to-treat approach.…”

“…The development of this system and its associated statistical considerations have been described elsewhere. 10 The system is illustrated in Figure 1 and briefly described here.…”

“…Control subgroup B and treatment subgroup B were equivalent except with respect to service; thus, control subgroup B served as a control for treatment subgroup B in the determination of the effect of counseling. 10 This randomization procedure did not affect the participants' ability to receive treatment, because all smokers, including those in the control group, received the same callback instructions, and those in the control group were not denied counseling if they did call back. Therefore, callers who wanted treatment were not discouraged from participating in the study, and we were able to recruit from the full spectrum of callers.…”