Are propensity scores really superior to standard multivariable analysis?

Abstract

Clinicians often face difficult decisions despite the lack of evidence from randomized trials. Thus, clinical evidence is often shaped by non-randomized studies exploiting multivariable approaches to limit the extent of confounding. Since their introduction, propensity scores have been used more and more frequently to estimate relevant clinical effects adjusting for established confounders, especially in small datasets. However, debate persists on their real usefulness in comparison to standard multivariable approaches such as logistic regression and Cox proportional hazard analysis. This holds even truer in light of key quantitative developments such as bootstrap and Bayesian methods. This qualitative review aims to provide a concise and practical guide to choose between propensity scores and standard multivariable analysis, emphasizing strengths and weaknesses of both approaches.

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

“…Our review identified a total of nine studies: eight studies 62-69 compared the results of regression analysis, instrumental variables and propensity scoring and a further study 70 discussed the relative merits of the alternative methods of adjustment. Two of these studies were systematic reviews: Shah et al 66 reviewed comparisons between propensity scoring and regression methods and Shah et al 66 and Laborde-Castérot et al 69 reviewed comparisons between propensity scoring and instrumental variable analyses.…”

“…Biondi-Zoccai et al 70 concluded that there is no clearly superior method, noting that 'both standard multivariable methods and propensity scores have key limitations, and none is able to take into account unknown confounders' (p. 736). Biondi-Zoccai et al, 70 however, go on to suggest that propensity scoring methods may have advantages over regression methods when the sample size is small and that, although instrumental variables methods are not without their limitations, they are the only methods that allow for unobserved confounding to be adjusted for.…”

“…In addition to the seven empirical studies identified, [62][63][64][65][66][67]69 a discussion paper by Biondi-Zoccai et al 70 provides a useful overview of the alternative methods of adjustment. Biondi-Zoccai et al 70 concluded that there is no clearly superior method, noting that 'both standard multivariable methods and propensity scores have key limitations, and none is able to take into account unknown confounders' (p. 736).…”

“…A summary of the studies that have looked at methods of adjustment for confounding bias in NRSs and how reliable they are [62][63][64][65][66][67][68][69][70] is presented in Appendix 3. Overall, it is unclear which methods are most appropriate in certain circumstances and further research is needed.…”

“…Although the presentation of population NHEs is not formally requested within the existing NICE methods guide, 122 population-based analyses are requested to be submitted to assess population impacts within Section 5.12 (Impact on the NHS). In addition, Section 6.4.1. states that in situations in which the evidence of clinical effectiveness is 'absent, weak or uncertain', the committee is requested to 'balance the potential net benefits to current NHS patients of a recommendation not restricted to research with the potential net benefits to both current and future NHS patients of being able to produce guidance and base clinical practice on a more secure evidence base' (p. [69][70][71]. 122 Analyses of population NHEs may therefore provide additional information to help inform the committee's deliberations regarding possible research recommendations (Section 6.4 of the current methods guide 122 ).…”