Transparency and reporting characteristics of COVID-19 randomized controlled trials

Abstract

Background In the context of the COVID-19 pandemic, randomized controlled trials (RCTs) are essential to support clinical decision-making. We aimed (1) to assess and compare the reporting characteristics of RCTs between preprints and peer-reviewed publications and (2) to assess whether reporting improves after the peer review process for all preprints subsequently published in peer-reviewed journals. Methods We searched the Cochrane COVID-19 Study Register and L·OVE COVID-19 platform to identify all reports of RCTs assessing pharmacological treatments of COVID-19, up to May 2021. We extracted indicators of transparency (e.g., trial registration, data sharing intentions) and assessed the completeness of reporting (i.e., some important CONSORT items, conflict of interest, ethical approval) using a standardized data extraction form. We also identified paired reports published in preprint and peer-reviewed publications. Results We identified 251 trial reports: 121 (48%) were first published in peer-reviewed journals, and 130 (52%) were first published as preprints. Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items. Conclusions Transparency, completeness, and consistency of reporting of COVID-19 clinical trials were insufficient both in preprints and peer-reviewed publications. A comparison of paired reports published in preprint and peer-reviewed publication did not indicate major improvement.

Dissemination of Registered COVID-19 Clinical Trials (DIRECCT): a cross-sectional study

“…While faster dissemination, via preprints or registries, does draw concerns around unvetted or low-quality results entering the public domain, it also allows high-impact results to be adopted into care more quickly [ 55 ]. Evidence has shown that COVID-19 preprints that convert to publications are typically concordant in their main characteristics [ 27 , 28 , 56 ] while those that remain unpublished tend to have more issues [ 29 ]. “Hot” topics like COVID-19 also likely draw more intense scrutiny during the pre-publication review process that will lead to public discussion around controversial or low-quality preprinted results [ 57 ].…”

Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials

“…A meta-research study of 139 studies reported in preprint and subsequent journal article or in different versions of the preprint found a change in the abstract’s conclusion in 24% of studies [ 7 ]. In contrast, a study of 78 preprint–article pairs of RCTs showed consistency in terms of the completeness of reporting [ 8 ]. Another analysis of 67 interventional and observational studies found that preprints and their subsequent journal articles were similar in terms of reporting and spin (i.e., distorted interpretation of results) [ 9 ].…”