The methodological quality of individual participant data meta-analysis on intervention effects: systematic review

Abstract

Objective To assess the methodological quality of individual participant data (IPD) meta-analysis and to identify areas for improvement. Design Systematic review. Data sources Medline, Embase, and Cochrane Database of Systematic Reviews. Eligibility criteria for selecting studies Systematic reviews with IPD meta-analyses of randomised controlled trials on intervention effects published in English. Results 323 IPD meta-analyses covering 21 clinical areas and published between 1991 and 2019 were included: 270 (84%) were non-Cochrane reviews and 269 (84%) were published in journals with a high impact factor (top quarter). The IPD meta-analyses showed low compliance in using a satisfactory technique to assess the risk of bias of the included randomised controlled trials (43%, 95% confidence interval 38% to 48%), accounting for risk of bias when interpreting results (40%, 34% to 45%), providing a list of excluded studies with justifications (32%, 27% to 37%), establishing an a priori protocol (31%, 26% to 36%), prespecifying methods for assessing both the overall effects (44%, 39% to 50%) and the participant-intervention interactions (31%, 26% to 36%), assessing and considering the potential of publication bias (31%, 26% to 36%), and conducting a comprehensive literature search (19%, 15% to 23%). Up to 126 (39%) IPD meta-analyses failed to obtain IPD from 90% or more of eligible participants or trials, among which only 60 (48%) provided reasons and 21 (17%) undertook certain strategies to account for the unavailable IPD. Conclusions The methodological quality of IPD meta-analyses is unsatisfactory. Future IPD meta-analyses need to establish an a priori protocol with prespecified data syntheses plan, comprehensively search the literature, critically appraise included randomised controlled trials with appropriate technique, account for risk of bias during data analyses and interpretation, and account for unavailable IPD.

Methodological Aspects of Meta-Analyses Assessing the Effect of Blood Pressure–Lowering Treatment on Clinical Outcomes

“…Indeed, a recent meta-epidemiological study found that a priori protocols more often are lacking for IPD meta-analyses compared with traditional study-level meta-analyses. 50 Furthermore, risk of bias for the included trials are less commonly assessed according to recommended standards, and only 40% of IPD meta-analyses accounted for risk of bias in any way when interpreting the overall results (See Summarizing the Evidence below). 50…”

“…50 Furthermore, risk of bias for the included trials are less commonly assessed according to recommended standards, and only 40% of IPD meta-analyses accounted for risk of bias in any way when interpreting the overall results (See Summarizing the Evidence below). 50…”

New horizons in evidence-based care for older people: individual participant data meta-analysis

“…Despite the challenges in obtaining, and preparing IPD for MA, a systematic review of 323 IPDMAs across different specialty areas reported that 54% obtained IPD from over 80% of eligible trials, and 87% obtained IPD from over 50% of eligible trials [ 25 ]. The 2021 United Nations Educational, Scientific and Cultural Organisation (UNESCO) Recommendation on Open Science provides strong global support for making scientific knowledge openly available, accessible and reusable [ 26 ].…”

The importance of adherence to international standards for depositing open data in public repositories

“…It has been highlighted that there are several main types of research data repositories: Institutional, disciplinary, multidisciplinary and project specific [ 10 ]. Availability of raw data in open repositories facilitates the adequate development of meta-analysis, particularly individual patient data -IPD- meta analyses [ 11 ], and the cumulative evaluation of evidence for specific topics [ 12 ], especially for high-dimensional data [ 13 ] (such as results from genomics, transcriptomics or epigenomics). In this context, certain research fields, such as genomics, have developed standards that facilitate and promote deposition of raw data [ 14 ].…”