A systematic examination of preprint platforms for use in the medical and biomedical sciences setting

Kirkham, Jamie; Penfold, Naomi; Murphy, Fiona; Boutron, Isabelle; Ioannidis, John P. A.; Moher, David

doi:10.1101/2020.04.27.063578

Cited by 34 publications

(43 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The large-scale use of preprints is a novel development for the biomedical sciences [13], and new mechanisms to manage this evidence dissemination track need to be invented. The algorithm designed for the present study offers such a mechanism by detecting the appearance of new evidence sources in real-time and summarizing this information in a web interface [14].…”

Section: Meaning Of the Studymentioning

confidence: 99%

Changes in evidence for studies assessing interventions for COVID-19 reported in preprints: meta-research study

Oikonomidi

Boutron

Pierre

et al. 2020

BMC Med

Self Cite

View full text Add to dashboard Cite

Background The increasing use of preprints to disseminate evidence on the effect of interventions for the coronavirus disease 2019 (COVID-19) can lead to multiple evidence sources for a single study, which may differ in the reported evidence. We aim to describe the proportion of evidence on the effect of interventions for COVID-19 from preprints and journal articles and map changes in evidence between and within different sources reporting on the same study. Methods Meta-research study. We screened the Cochrane living systematic review and network meta-analysis (COVID-NMA) database to identify all preprints and journal articles on all studies assessing interventions for COVID-19 published up to 15 August 2020. We compared all evidence sources (i.e., preprint and associated journal article) and the first and latest versions of preprints for each study to identify changes in two evidence components: study results (e.g., numeric change in hazard ratio, odds ratio, event rate, or change in p value > or < 0.05 in any outcome) and abstract conclusions (classified as positive, negative or neutral regarding the intervention effect, and as reporting uncertainty in the findings or not). Changes in study results were further classified as important changes if they (1) represented a change in any effect estimate by ≥ 10% and/or (2) led to a change in the p value crossing the threshold of 0.05. Results We identified 556 studies. In total, 338 (61%) had been reported in a preprint: 66 (20%) of these had an associated journal article (median time to publication 76 days [interquartile range (IQR) 55–106]) and 91 (27%) had > 1 preprint version. A total of 139 studies (25% of the overall sample) were reported in multiple evidence sources or versions of the same source: for 63 (45%), there was a change in at least one evidence component between or within sources (42 [30%] had a change in study results, and in 29 [21%] the change was classified as important; 33 [24%] had a change in the abstract conclusion). For studies with both a preprint and an article, a median of 29% (IQR 14–50) of total citations were attributed to the preprint instead of the article. Conclusions Results on the effect of interventions for COVID-19 are often reported in multiple evidence sources or source versions for a single study. Evidence is not stable between and within evidence sources. Real-time linkage of all sources per study could help to keep systematic reviews up-to-date.

show abstract

Section: Meaning Of the Studymentioning

confidence: 99%

Changes in evidence for studies assessing interventions for COVID-19 reported in preprints: meta-research study

Oikonomidi

Boutron

Pierre

et al. 2020

BMC Med

Self Cite

View full text Add to dashboard Cite

show abstract

“…The copyright holder for this preprint this version posted September 7, 2020. ; https://doi.org/10.1101/2020.09.04.20188771 doi: medRxiv preprint in the vetting process (Kirkham et al, 2020). Although not to be mistaken for a formal pre-ppr, it does indicate that most biomedical preprints vet preprints for quality-control before officially make manuscripts available on their repositories.…”

Section: Competition and Epistemic Oversupplymentioning

confidence: 99%

COVID-19 Preprints and Their Publishing Rate: An Improved Method

Lachapelle

2020

Preprint

View full text Add to dashboard Cite

Context: As the COVID-19 pandemic persists around the world, the scientific community continues to produce and circulate knowledge on the deadly disease at an unprecedented rate. During the early stage of the pandemic, preprints represented nearly 40% of all English-language COVID-19 scientific corpus (6, 000+ preprints | 16, 000+ articles). As of mid-August 2020, that proportion dropped to around 28% (13, 000+ preprints | 49, 000+ articles). Nevertheless, preprint servers remain a key engine in the efficient dissemination of scientific work on this infectious disease. But, giving the uncertified nature of the scientific manuscripts curated on preprint repositories, their integration to the global ecosystem of scientific communication is not without creating serious tensions. This is especially the case for biomedical knowledge since the dissemination of bad science can have widespread societal consequences. Scope: In this paper, I propose a robust method that will allow the repeated monitoring and measuring of COVID-19 preprint's publication rate. I also introduce a new API called Upload-or-Perish. It is a micro-API service that enables a client to query a specific preprint manuscript's publication status and associated meta-data using a unique ID. This tool is in active development. Data: I use Covid-19 Open Research Dataset (CORD-19) to calculate COVID-19 preprint corpus' conversion rate to peer-reviewed articles. CORD-19 dataset includes preprints from arXiv, bioRxiv, and medRxiv. Methods: I utilize conditional fuzzy logic on article titles to determine if a preprint has a published counterpart version in the database. My approach is an important departure from previous studies that rely exclusively on bioRxiv API to ascertain preprints' publication status. This is problematic since the level of false positives in bioRxiv metadata could be as high as 37%. Findings: My analysis reveals that around 15% of COVID-19 preprint manuscripts in CORD-19 dataset that were uploaded on from arXiv, bioRxiv, and medRxiv between January and early August 2020 were published in a peer-reviewed venue. When compared to the most recent measure available, this represents a two-fold increase in a period of two months. My discussion review and theorize on the potential explanations for COVID-19 preprints' low conversion rate.

show abstract

“…The studies will be written up for publication in open-access, peer-reviewed journals, prioritising outlets which will facilitate a speedy publication timeframe. Preprint versions of the manuscripts will be posted on public preprint servers ( Kirkham et al , 2020 ) before formal publication in scholarly journals. Relevant reporting guidelines will be utilised to ensure the completeness and transparency of the articles; specifically, findings from WP1 will be reported in line with the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement ( von Elm et al , 2008 ), and findings from WP2 will be reported in line with the COnsolidated criteria for REporting Qualitative research (COREQ) checklist ( Tong et al , 2007 ).…”

Section: Protocolmentioning

confidence: 99%

Identifying and addressing psychosocial determinants of adherence to physical distancing guidance during the COVID-19 pandemic – project protocol

et al. 2020

View full text Add to dashboard Cite

Optimising public health physical distancing measures has been a critical part of the global response to the spread of COVID-19. Evidence collected during the current pandemic shows that the transmission rate of the virus is significantly reduced following implementation of intensive physical distancing measures. Adherence to these recommendations has been poorer than adherence to other key transmission reduction behaviours such as handwashing. There are a complex range of reasons that are likely to predict why people do not or only partially adhere to physical distancing recommendations. In the current project we aim to address the following research questions: (1) What are the psychosocial determinants of physical distancing for the general public and for key socio-demographic sub-groups (e.g., young adults, older adults, etc.)?; (2) Do current Government of Ireland COVID-19 physical distancing communications address the determinants of physical distancing?; and (3) How can communications be optimised and tailored to sub-groups to ensure maximum adherence to guidelines? These will be addressed by conducting three work packages (WPs). In WP1, we will work closely with the iCARE international study, which includes a large online survey of public responses to measures established to reduce and slow the spread of COVID-19, including physical distancing. We will analyse Irish data, comparing it to data from other countries, to identify the key psychosocial determinants of physical distancing behaviour. This will be followed by a qualitative study to explore in depth the barriers and facilitators of physical distancing behaviour among the Irish public (WP2). In WP3, we will conduct a content analysis and evidence mapping of current government messaging around physical distancing, to ensure the findings from this research feed into the development of ongoing communication and future messaging about physical distancing.

show abstract

A systematic examination of preprint platforms for use in the medical and biomedical sciences setting

Cited by 34 publications

References 11 publications

Changes in evidence for studies assessing interventions for COVID-19 reported in preprints: meta-research study

Changes in evidence for studies assessing interventions for COVID-19 reported in preprints: meta-research study

COVID-19 Preprints and Their Publishing Rate: An Improved Method

Identifying and addressing psychosocial determinants of adherence to physical distancing guidance during the COVID-19 pandemic – project protocol

Contact Info

Product

Resources

About