Implications of the Propensity Score Matching Paradox in Pharmacoepidemiology

Abstract

Recent work has demonstrated that propensity score matching may lead to increased covariate imbalance, even with the corresponding decrease in propensity score distance between matched units. The extent to which this paradoxical phenomenon might harm causal inference in real epidemiologic studies has not been explored. We evaluated the effect of this phenomenon using insurance claims data from the Pharmaceutical Assistance Contract for the Elderly (1999-2002) and Medicaid Analytic eXtract (2000-2007) databases in the United States. For each data set, we created several 1:1 propensity-score-matched data sets by manipulating the size of the covariate set used to generate propensity scores, the index exposure prevalence in the prematched data set, and the matching algorithm. We matched all index units, then progressively pruned matched sets in order of decreasing propensity score distance, calculating covariate imbalance after each pruning. Although covariate imbalance sometimes increased after progressive pruning of matched sets, the application of commonly used propensity score calipers for defining an acceptable match stopped pruning near the lowest region of the imbalance trend and resulted in an improvement over the imbalance in the prematched data set. Thus, propensity score matching does not appear to induce increased covariate imbalance when standard propensity score calipers are applied in these types of pharmacoepidemiologic studies.

Empagliflozin and the Risk of Heart Failure Hospitalization in Routine Clinical Care

“…(6) Patients were 1:1 PS-matched using the nearest neighbor methodology with a maximum caliper of 0.01 of the PS. (7,8) Post-matching covariate balance between treatments was assessed for each covariate by the calculation of standardized differences, i.e., the difference in means or proportions divided by the pooled standard deviation, with meaningful imbalances set at values greater than 0.1. (9, 10) Hazard ratios (HRs) and 95% confidence intervals (CI) were estimated in each data source and pooled across the data sources using a fixed-effects metaanalysis, 7 since random effects pooling can be biased in the context of few databases.…”

“…(7,8) Post-matching covariate balance between treatments was assessed for each covariate by the calculation of standardized differences, i.e., the difference in means or proportions divided by the pooled standard deviation, with meaningful imbalances set at values greater than 0.1. (9, 10) Hazard ratios (HRs) and 95% confidence intervals (CI) were estimated in each data source and pooled across the data sources using a fixed-effects metaanalysis, 7 since random effects pooling can be biased in the context of few databases. 8 In order to address potential unmeasured confounding, we conducted the following sensitivity analyses -(1) we performed 1:1 high-dimensional propensity score (hdPS) matching, which enriched the original PS with 100 additional empirically identified covariates; (11) and 2we assessed the association with a control outcome with an expected null finding, i.e., the occurrence of flu vaccination during follow-up.…”

Over-the-scope clip vs epinephrine with clip for first-line hemostasis in non-variceal upper gastrointestinal bleeding: a propensity score match analysis

“…In our study, the post-hoc power (84 patients in each group, with a 0.05 alpha error and rebleeding rates of 20 % in the conventional group and 8 % in the OTSC group) is 61.2 %, which is slightly underpowered in comparison to the standard reference value (80 %). Second, propensity score matching may lead to increased covariate imbalance called the propensity score paradox [27]. Despite progressive pruning of the matched sets, the application of a caliper width of 0.1 should avoid pruning near the lowest region of the imbalance trend.…”

CF290 for pancolonic chromoendoscopy improved sessile serrated polyp detection and procedure time: a propensity score-matching study

“…Fourth, propensity score model overfitting has been reported to lead to inflated variances of the estimated odds ratios 28 29 . Furthermore, adjustment using propensity score matching has recently been criticized as a paradoxically problematic method 30 . Propensity score-matching in our study might have resulted in some biases…”