Evaluation of the Propensity score methods for estimating marginal odds ratios in case of small sample size

Abstract

BackgroundPropensity score (PS) methods are increasingly used, even when sample sizes are small or treatments are seldom used. However, the relative performance of the two mainly recommended PS methods, namely PS-matching or inverse probability of treatment weighting (IPTW), have not been studied in the context of small sample sizes.MethodsWe conducted a series of Monte Carlo simulations to evaluate the influence of sample size, prevalence of treatment exposure, and strength of the association between the variables and the outcome and/or the treatment exposure, on the performance of these two methods.ResultsDecreasing the sample size from 1,000 to 40 subjects did not substantially alter the Type I error rate, and led to relative biases below 10%. The IPTW method performed better than the PS-matching down to 60 subjects. When N was set at 40, the PS matching estimators were either similarly or even less biased than the IPTW estimators. Including variables unrelated to the exposure but related to the outcome in the PS model decreased the bias and the variance as compared to models omitting such variables. Excluding the true confounder from the PS model resulted, whatever the method used, in a significantly biased estimation of treatment effect. These results were illustrated in a real dataset.ConclusionEven in case of small study samples or low prevalence of treatment, PS-matching and IPTW can yield correct estimations of treatment effect unless the true confounders and the variables related only to the outcome are not included in the PS model.

Morphine Is Associated With a Delayed Activity of Oral Antiplatelet Agents in Patients With ST-Elevation Acute Myocardial Infarction Undergoing Primary Percutaneous Coronary Intervention

“…Variables that were significantly different between the 2 groups and those that are known to affect platelet reactivity were incorporated in the model: age (years), sex, body mass index, diabetes mellitus, smoking, killip class, systolic blood pressure, heart rate, anterior infarct location, and bivalirudin. 9,10 Model discrimination was assessed with the c-statistic and goodness of fit with the Hosmer-Lemeshow test. Thereafter, a logistic regression analysis was performed to adjust HRPR for the propensity score used as a continuous covariate.…”

Adherence and Persistence Among Chronic Myeloid Leukemia Patients During Second-Line Tyrosine Kinase Inhibitor Treatment

“…Studies have shown that propensity score matching and IPTW remove systematic differences between subjects in different treatment groups to a greater degree than propensity score stratification or covariate adjustment using the propensity score. 18,19 One particular study showed that IPTW performed better than propensity score matching with sample sizes as low as 60. In light of this evidence and in order to retain the original sample, IPTW was utilized, rather than propensity score matching.…”

Can private food standards promote gender equality in the small farm sector?

“…To reduce such bias, we use a technique called "entropy balancing" (Hainmueller, 2012). Entropy balancing belongs to the family of weighting and matching approaches, such as inverse probability weighting (IPW) and propensity score matching (PSM) (Hirano et al, 2003;Pirracchio et al, 2012). Weighting and matching approaches are used to address systematic differences (imbalances) in the distribution of covariates between the treatment group (in our case certified households) and the control group (in our case non-certified households).…”

“…Weighting and matching procedures can produce unbiased treatment effects when relevant confounding factors are controlled for and the number of observations is sufficiently large (Hirano et al, 2003;Pirracchio et al, 2012;Wooldridge, 2007). What exactly "sufficiently large" means depends on the context and the actual distribution of treated and control observations.…”