Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation

Bero, Lisa; Lawrence, Rosa; Leslie, Louis; Chiu, Kellia; McDonald, Sally; Page, Matthew J.; Grundy, Quinn; Parker, Lisa; Boughton, Stephanie; Kirkham, Jamie J; Featherstone, Robin

doi:10.1136/bmjopen-2021-051821

Cited by 36 publications

(32 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pagliaro manually examined the full text of 10 preprints in chemistry, finding only small changes in this sample [ 16 ], and Kataoka compared the full text of medRxiv randomised controlled trials (RCTs) related to COVID-19, finding in preprint versions an increased rate of spin (positive terms in the title or abstract conclusion section used to describe nonsignificant results [ 17 ]. Bero and colleagues [ 18 ] and Oikonomidi and colleagues [ 19 ] investigated changes in conclusions reported in COVID-19–related clinical studies, finding that some preprints and journal articles differed in the outcomes reported. However, the frequency of changes in the conclusions of a more general sample of preprints remained an open question.…”

Section: Introductionmentioning

confidence: 99%

Tracking changes between preprint posting and journal publication during a pandemic

et al. 2022

View full text Add to dashboard Cite

Amid the Coronavirus Disease 2019 (COVID-19) pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press, and policymakers for the first time. This phenomenon has sharpened long-standing questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19–related and 17.2% of COVID-19–related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.

show abstract

Section: Introductionmentioning

confidence: 99%

Tracking changes between preprint posting and journal publication during a pandemic

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However, our results do not support the hypothesis that peer-reviewed journal publications are of better quality compared to preprints. We found no difference in terms of transparency and reporting between the preprint and the peer-reviewed report [17, 18].…”

Section: Discussionmentioning

confidence: 99%

Transparency and reporting characteristics of COVID-19 randomized controlled trials

Kapp

Esmail

Ghosn

et al. 2022

Preprint

View full text Add to dashboard Cite

Randomized controlled trials (RCTs) are essential to support clinical decision making. We assessed the transparency, completeness and consistency of reporting of 244 reports (120 peer-reviewed journal publications; 124 preprints) of RCTs assessing pharmacological interventions for the treatment of COVID-19 published the first 17 months of the pandemic (up to May 31, 2021). Transparency was poor. Only 55% of trials were prospectively registered; 39% made their full protocols available and 29% provided access to their statistical analysis plan. Only 6% completely reported the most important information. Primary outcome(s) reported in trial registries and published reports were inconsistent in 47% of trials. Of the 124 RCTs published as preprint, 76 were secondarily published in a peer-reviewed journal. There was no major improvement after the peer-review process. Lack of transparency, completeness and consistency of reporting is an important barrier to trust, interpretation and synthesis in COVID-19 clinical trials.

show abstract

“…However, when studying 67 studies on Covid-19 prevention or treatment, Bero et al . found only 23 articles (34%) that had no discrepancies in results reporting between preprints and journal publications [8]. The changes in the conclusions were less frequent than the changes in studies’ result, mostly because conclusions were usually broad, such as “among patients hospitalized with Covid-19, those who received hydroxychloroquine did not have a lower incidence of death at 28 days than those who received usual care” [22].…”

Section: Discussionmentioning

confidence: 99%

“…Only a small proportion will be converted to scientific publications and the share of converted preprints shows a declining trend over time [4][5][6][7]. For those finally passing the peer-review process, peer-reviewers' comments and critiques lead authors to revise their manuscripts, substantially at times [4,6,8]. In that case, the preprint is not the latest version of the work anymore and identifying if it has been published, and referring to the published version instead of the preprint is a major issue as it provides readers with the latest version of a now certified work.…”

Section: Introductionmentioning

confidence: 99%

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals

Gehanno

Grosjean

Darmoni

et al. 2022

Preprint

View full text Add to dashboard Cite

Background : Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version. Methods : Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version. Results : Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints. Conclusions : More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

show abstract

Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation

Cited by 36 publications

References 34 publications

Tracking changes between preprint posting and journal publication during a pandemic

Tracking changes between preprint posting and journal publication during a pandemic

Transparency and reporting characteristics of COVID-19 randomized controlled trials

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals

Contact Info

Product

Resources

About