Generalizing Evidence from Randomized Trials Using Inverse Probability of Sampling Weights

Abstract

Summary Results obtained in randomized trials may not easily generalize to target populations. Whereas in randomized trials the treatment assignment mechanism is known, the sampling mechanism by which individuals are selected to participate in the trial is typically not known and assuming random sampling from the target population is often dubious. We consider an inverse probability of sampling weighted (IPSW) estimator for generalizing trial results to a target population. The IPSW estimator is shown to be consistent and asymptotically normal. A consistent sandwich-type variance estimator is derived and simulation results are presented comparing the IPSW estimator to a previously proposed stratified estimator. The methods are then utilized to generalize results from two randomized trials of HIV treatment to all people living with HIV in the United States.

Doubly Robust Inference when Combining Probability and Non-Probability Samples with High Dimensional Data

“…() and Buchanan et al . () used propensity score weighting to generalize results from randomized trials to a target population. O’Muircheartaigh and Hedges () proposed propensity score stratification for analysing a non‐randomized social experiment.…”

“…The subsequent adjustments, such as propensity score weighting or stratification, can then be used to adjust for selection biases; see, for example, Lee and Valliant (2009), Valliant and Dever (2011), Elliott and Valliant (2017) and Chen, Li and Wu (2018). Stuart et al (2011Stuart et al ( , 2015 and Buchanan et al (2018) used propensity score weighting to generalize results from randomized trials to a target population. O'Muircheartaigh and Hedges (2014) proposed propensity score stratification for analysing a non-randomized social experiment.…”

GIST 2.0: A scalable multi-trait metric for quantifying population representativeness of individual clinical studies

“…Currently, the lack of population representativeness in clinical studies remains largely undiscovered until after study publications (e.g. [12]-details in Sec. 1.2).…”

“…As mentioned above, most of the generalizability assessments have been performed after the completion of a trial. For example, in a technical report by Buchanan et al [12], a generalizability study was performed for HIV treatment clinical trials. The majority of the results presented in this study were simulation-based and only two clinical trials were evaluated.…”

Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials

“…Moreover, many of these compute sample-driven (rather than eligibility driven) generalizability after the publication of study results. [29][30][31] In a study by Bleeker et al, 32 eligibility-driven generalizability was computed using receiver operating characteristic analysis. Specifically, infants with fever were evaluated for the presence of bacterial infection by a binary classifier, and training and testing sets were composed of patients from two different hospitals in different time periods.…”