Exploring changes over time and characteristics associated with data retrieval across individual participant data meta-analyses: systematic review

Abstract

ObjectiveTo investigate whether the success rate of retrieving individual participant data (IPD) for use in IPD meta-analyses has increased over time, and to explore the characteristics associated with IPD retrieval.DesignSystematic review of published IPD meta-analyses, supplemented by a reflection of the Cochrane Epilepsy Group’s 20 years’ experience of requesting IPD.Data sourcesMedline, CENTRAL, Scopus, Web of Science, CINAHL Plus, and PsycINFO.Eligibility criteria for study selectionIPD meta-analyses of studies of all designs and all clinical areas published in English.Results760 IPD meta-analyses which identified studies by systematic methods that had been published between 1987 and 2015 were included. Only 188 (25%) of these IPD meta-analyses retrieved 100% of the eligible IPD for analysis, with 324 (43%) of these IPD meta-analyses retrieving 80% or more of relevant IPD. There is insufficient evidence to suggest that IPD retrieval rates have improved over time. IPD meta-analyses that included only randomised trials, had an authorship policy, included fewer eligible participants, and were conducted outside of the Cochrane Database of Systematic Reviews were associated with a high or complete IPD retrieval rate. There was no association between the source of funding of the IPD meta-analyses and IPD retrieval rate. The IPD retrieval rate of the Cochrane Epilepsy Group has declined from 83% (up to 2005) to 65% (between 2012 and 2015) and the reported reasons for lack of data availability have changed in recent years.ConclusionsIPD meta-analyses are considered to be the “gold standard” for the synthesis of data from clinical research studies; however, only 25% of published IPD meta-analyses have had access to all IPD.

Use of multiple covariates in assessing treatment‐effect modifiers: A methodological review of individual participant data meta‐analyses

“…First, we included a pre‐determined sample of 100 IPD meta‐analyses, and we did not assess an additional 200–300 IPD meta‐analyses of randomised trials between 2015 and 2020. However, we do believe that this is a representative sample as our sample is comparable to IPD meta‐analyses identified in a systematic review 26 for a number of characteristics such as clinical area, number of included studies and number of participants included. Also, some IPD meta‐analyses only investigated one treatment‐covariate interaction and others may not have found evidence of multiple effect modifiers.…”

Use of multiple covariates in assessing treatment‐effect modifiers: A methodological review of individual participant data meta‐analyses

“…First, we included a pre‐determined sample of 100 IPD meta‐analyses, and we did not assess an additional 200–300 IPD meta‐analyses of randomised trials between 2015 and 2020. However, we do believe that this is a representative sample as our sample is comparable to IPD meta‐analyses identified in a systematic review 26 for a number of characteristics such as clinical area, number of included studies and number of participants included. Also, some IPD meta‐analyses only investigated one treatment‐covariate interaction and others may not have found evidence of multiple effect modifiers.…”

Neutrophil‐to‐lymphocyte ratio as a prognostic marker for head and neck squamous cell carcinoma treated with immune checkpoint inhibitors: Meta‐analysis

“…However, the IPDMA will very often face low data retrieval rate, resulting in availability bias. 48 In conclusion, the NLR is a prognostic marker of ICI treatment in patients with HNSCC. Further research is required on how the NLR assessment should be incorporated into clinical practice.…”

State of play in individual participant data meta-analyses of randomised trials: Systematic review and consensus-based recommendations

“…This apparent difference may have several explanations. First, by including only IPD meta-analyses that were successfully completed and published, we may have missed those abandoned due to difficulties obtaining sufficient IPD in a suitable timeframe (29), which is supported by examples shared by our panel and in published reports (30). Second, our data extraction relied on the self-reporting of IPD meta-analysis authors; which may not always be accurate.…”

Use of multiple covariates in assessing treatment‐effect modifiers: A methodological review of individual participant data meta‐analyses

“…First, we included a pre‐determined sample of 100 IPD meta‐analyses, and we did not assess an additional 200–300 IPD meta‐analyses of randomised trials between 2015 and 2020. However, we do believe that this is a representative sample as our sample is comparable to IPD meta‐analyses identified in a systematic review 26 for a number of characteristics such as clinical area, number of included studies and number of participants included. Also, some IPD meta‐analyses only investigated one treatment‐covariate interaction and others may not have found evidence of multiple effect modifiers.…”

Neutrophil‐to‐lymphocyte ratio as a prognostic marker for head and neck squamous cell carcinoma treated with immune checkpoint inhibitors: Meta‐analysis

“…However, the IPDMA will very often face low data retrieval rate, resulting in availability bias. 48 In conclusion, the NLR is a prognostic marker of ICI treatment in patients with HNSCC. Further research is required on how the NLR assessment should be incorporated into clinical practice.…”

State of play in individual participant data meta-analyses of randomised trials: Systematic review and consensus-based recommendations

“…This apparent difference may have several explanations. First, by including only IPD meta-analyses that were successfully completed and published, we may have missed those abandoned due to difficulties obtaining sufficient IPD in a suitable timeframe (29), which is supported by examples shared by our panel and in published reports (30). Second, our data extraction relied on the self-reporting of IPD meta-analysis authors; which may not always be accurate.…”

Use of multiple covariates in assessing treatment‐effect modifiers: A methodological review of individual participant data meta‐analyses

“…First, we included a pre‐determined sample of 100 IPD meta‐analyses, and we did not assess an additional 200–300 IPD meta‐analyses of randomised trials between 2015 and 2020. However, we do believe that this is a representative sample as our sample is comparable to IPD meta‐analyses identified in a systematic review 26 for a number of characteristics such as clinical area, number of included studies and number of participants included. Also, some IPD meta‐analyses only investigated one treatment‐covariate interaction and others may not have found evidence of multiple effect modifiers.…”

Neutrophil‐to‐lymphocyte ratio as a prognostic marker for head and neck squamous cell carcinoma treated with immune checkpoint inhibitors: Meta‐analysis

“…However, the IPDMA will very often face low data retrieval rate, resulting in availability bias. 48 In conclusion, the NLR is a prognostic marker of ICI treatment in patients with HNSCC. Further research is required on how the NLR assessment should be incorporated into clinical practice.…”

State of play in individual participant data meta-analyses of randomised trials: Systematic review and consensus-based recommendations

“…This apparent difference may have several explanations. First, by including only IPD meta-analyses that were successfully completed and published, we may have missed those abandoned due to difficulties obtaining sufficient IPD in a suitable timeframe (29), which is supported by examples shared by our panel and in published reports (30). Second, our data extraction relied on the self-reporting of IPD meta-analysis authors; which may not always be accurate.…”