A Systematic Review of Conflicting Meta-Analyses in Orthopaedic Surgery

Abstract

Meta-analyses are important evaluations in orthopaedic surgery, not only to create clinical guidelines, but also because their findings are included in public health and health policy decision making. However, with increasing numbers of meta-analyses, discordant and frankly conflicting conclusions have been reported. We searched for conflicting meta-analyses, ie, those arriving at different conclusions despite following the same research question, identified potential reasons for these differences, and assessed the statistical significance and clinical importance of differences. We identified conflicting meta-analyses on graft choice in ACL reconstruction and the use of hyaluronic acid. We found significant differences in individual results only for meta-analyses on hyaluronic acid, but the 95% confidence intervals of the magnitude of differences included values as much as 40% for ACL meta-analyses. However, our findings suggest most conflicts derive from differences in the interpretation of pooled results rather than in the actual results. Thus conclusions and interpretations from meta-analyses should be scrutinized as critically as those from any other type of study and subjected to reassessment if deemed necessary.

Does Minimally Invasive Surgery Improve Short-term Recovery in Total Knee Arthroplasty?

“…To pool data, we used a random effects model to control for increased study heterogeneity. The more conservative random effects model involves an assumption that the effects being estimated in the different studies follow some distribution, but results in wider confidence intervals [11,44,45].…”

Discordances originated by multiple meta-analyses on interventions for myocardial infarction: a systematic review

“…Few other studies, mainly anecdotic, have investigated sources of discordance among SRs: different interpretation of identical quantitative results [8]; different interpretation due to noncomparable analyses because of, mainly unmotivated, subgroup analyses; different selection of primary or secondary outcomes [6]; different inclusion criteria [6]; and different meta-analytical approaches [9,28]. Our systematic attempt to explore the theoretical model proposed by Jadad et al [5], although limited to the cardiologic field, confirms many of the sources of discordance hypothesized.…”

Anterior cruciate ligament reconstruction results in alterations in gait variability

“…Biomechanical evaluations have demonstrated that differences do exist between the two autografts but they have not reached a consensus regarding the superiority of one autograft over the other in restoring neuromuscular function at the knee joint [10,12,13]. This is most evident in systematic reviews that have conflicting results as each favour one or the other graft [14]. It is possible that the examination of gait variability could better elucidate which autograft (BPTB or ST/G) is more suitable for ACL reconstruction.…”