Greedy caliper propensity score matching can yield variable estimates of the treatment‐outcome association—A simulation study

Abstract

Purpose: Greedy caliper propensity score (PS) matching is dependent on randomness, which can ultimately affect causal estimates. We sought to investigate the variation introduced by this randomness.Methods: Based on a literature search to define the simulation parameters, we simulated 36 cohorts of different sizes, treatment prevalence, outcome prevalence, treatment-outcome-association. We performed 1:1 caliper and nearest neighbor (NN) caliper PS-matching and repeated this 1000 times in the same cohort, before calculating the treatment-outcome association.Results: Repeating caliper and NN caliper matching in the same cohort yielded large variations in effect estimates, in all 36 scenarios, with both types of matching. The largest variation was found in smaller cohorts, where the odds ratio (OR) ranged from 0.53 to 10.00 (IQR of ORs: 1.11-1.67). The 95% confidence interval was not consistently overlapping a neutral association after repeating the matching with both algorithms. We confirmed these findings in a noninterventional example study. Conclusion:Caliper PS-matching can yield highly variable estimates of the treatment-outcome association if the analysis is repeated.

Risk for hypoglycemic emergency with levofloxacin use, a population-based propensity score matched nested case-control study

“…The scores were calculated by the logistic regression, including age, sex, household income, and baseline comorbidities (not including the DM severity (aDCSI score) and individual medications). One case was matched with 10 controls, according to the "nearest neighbor matching" (also called "greedy matching") [ 26 , 27 ]. Thus, the study dataset was constructed that comprised of PS matched case and control subjects.…”

Practice patterns and outcomes associated with intravenous albumin in patients with cirrhosis and acute kidney injury

“…We did not match for cirrhosis‐related complications and infections as the timing of these events was uncertain (ie, we did not want to match for complications that arose after AKI onset and treatment). Albumin users (albumin ± crystalloid) were matched by propensity score to non‐users (crystalloid only) with a greedy matching algorithm 32 . We matched first at 5 decimal places, then to 4 decimal places and then finally to 1 decimal place.…”

“…Albumin users (albumin ± crystalloid) were matched by propensity score to non-users (crystalloid only) with a greedy matching algorithm. 32 We matched first at 5 decimal places, then to 4 decimal places and then finally to 1 decimal place. We performed a 1:1 match, where the best match for each albumin user was randomly selected with each non-user once.…”

Research and scholarly methods: Propensity scores

“…Rank ordering by highest to lowest propensity score, lowest to highest propensity score, or random order (listed as examples) can affect who is matched to whom and qualitatively alter conclusions. 30 Third, will you form sets using optimal or greedy matching? In optimal matching, sets are formed to minimize the total within-set difference of the propensity score.…”

“…Second, how will you order participants before initiating the matching algorithm? Rank ordering by highest to lowest propensity score, lowest to highest propensity score, or random order (listed as examples) can affect who is matched to whom and qualitatively alter conclusions 30 . Third, will you form sets using optimal or greedy matching?…”