Value and usability of unpublished data sources for systematic reviews and network meta-analyses

Abstract

Peer-reviewed publications and conference proceedings are the mainstay of data sources for systematic reviews and network meta-analyses (NMA), but access to informative unpublished data is now becoming commonplace. To explore the usefulness of three types of 'grey' literature-clinical trials registries, clinical study reports and data from regulatory authorities-we conducted four case studies. The reporting of outcome data in peer-reviewed publications, the clinical trials registries and the clinical study reports for two clinical trials-one in melanoma, one in juvenile idiopathic arthritis (JIA)-was examined. In addition, we assessed the value of including unpublished data from the European Medicines Agency (EMA) and US Food and Drug Administration (FDA) in evidence syntheses of hepatitis C virus (HCV) and chronic obstructive pulmonary disease (COPD), respectively. For the clinical trials in melanoma and JIA, we identified outcome parameters on ClinicalTrials.gov additional to those reported in the peer-reviewed publications: subgroup data and additional efficacy end points/extended follow-up, respectively. The clinical study report also provided results for several subgroups unavailable elsewhere. For HCV and COPD, additional outcome data were obtained from the EMA European Public Assessment Report (EPAR) and the FDA, respectively, including data on subgroups and mortality. We conclude that data from these grey literature sources have the potential to influence results of systematic reviews and NMAs, and may thus have implications for healthcare decisions. However, it is important to consider carefully the availability, reliability and consequent usability of these data sources in systematic reviews and NMAs.

JAK-inhibitor and type I interferon ability to produce favorable clinical outcomes in COVID-19 patients: a systematic review and meta-analysis

“…The inclusion of pre-published work in meta-analyses has long been debated but are currently of utmost importance to push for larger randomized clinical trials during this critical time. Their inclusion may help attenuate artificially extreme effect estimates and counteract publication bias, but may leave the study vulnerable to further bias and misrepresentation [45,46]. For that reason, all included studies, published and pre-published, were individually reviewed in the evaluation of bias according to randomization status.…”

Rapid network meta-analysis using data from Food and Drug Administration approval packages is feasible but with limitations

“…In addition, disagreements were found across data sources in eligibility criteria, primary outcome, and primary analysis by our study and other studies. 11,[52][53][54][55][56][57] With inconsistent eligibility criteria, it is difficult for health care professionals and researchers to interpret and apply the trial findings. The inconsistency in describing the primary outcomes and the primary analyses is also problematic because results could be selectively reported by trialists to better align with their hypotheses.…”

A Narrative Review and Proposed Framework for Using Health System Data with Systematic Reviews to Support Decision-making

“…In most instances, these examples were explicit about their rationale for incorporating unpublished data, primarily because the published data was sparse (i.e., to increase certainty of findings by addressing strength of evidence), and/ or to determine whether the published data were applicable to health system populations (i.e., to increase the certainty of findings by addressing the applicability of evidence). [6][7][8][9][10][11][12][13][14][15][16][17][18][19] We identified several examples from the Mayo Clinic that illustrate different reasons for combining unpublished local with published data. In two such instances, published data for outcomes on uncommon procedures (e.g., total pancreatectomy, endovascular treatment carotid artery bifurcation aneurysms) were sparse, and adding unpublished local data increased the sample size and improved precision.…”