General medical publications during COVID-19 show increased dissemination despite lower validation

Abstract

Background The COVID-19 pandemic has yielded an unprecedented quantity of new publications, contributing to an overwhelming quantity of information and leading to the rapid dissemination of less stringently validated information. Yet, a formal analysis of how the medical literature has changed during the pandemic is lacking. In this analysis, we aimed to quantify how scientific publications changed at the outset of the COVID-19 pandemic. Methods We performed a cross-sectional bibliometric study of published studies in four high-impact medical journals to identify differences in the characteristics of COVID-19 related publications compared to non-pandemic studies. Original investigations related to SARS-CoV-2 and COVID-19 published in March and April 2020 were identified and compared to non-COVID-19 research publications over the same two-month period in 2019 and 2020. Extracted data included publication characteristics, study characteristics, author characteristics, and impact metrics. Our primary measure was principal component analysis (PCA) of publication characteristics and impact metrics across groups. Results We identified 402 publications that met inclusion criteria: 76 were related to COVID-19; 154 and 172 were non-COVID publications over the same period in 2020 and 2019, respectively. PCA utilizing the collected bibliometric data revealed segregation of the COVID-19 literature subset from both groups of non-COVID literature (2019 and 2020). COVID-19 publications were more likely to describe prospective observational (31.6%) or case series (41.8%) studies without industry funding as compared with non-COVID articles, which were represented primarily by randomized controlled trials (32.5% and 36.6% in the non-COVID literature from 2020 and 2019, respectively). Conclusions In this cross-sectional study of publications in four general medical journals, COVID-related articles were significantly different from non-COVID articles based on article characteristics and impact metrics. COVID-related studies were generally shorter articles reporting observational studies with less literature cited and fewer study sites, suggestive of more limited scientific support. They nevertheless had much higher dissemination.

The rapid, massive growth of COVID-19 authors in the scientific literature

“…However, many authors publishing on COVID-19 may have lacked proper background expertise. The explosive focus on COVID-19 may have caused some inappropriate ‘covidization’ of research [1,2], and the resulting research, conducted in such haste, may suffer from low validity [3,4].…”

Translation in action: Influence, collaboration, and evolution of COVID‐19 research with Clinical and Translational Science Awards consortium support

“…Still, concerns have arisen that fast-tracked publishing, during a time of intense need for information sharing, may have resulted in lower quality research (and inflated citation rates) by authors working outside their expertise and publishing without sufficient peer review. [23][24][25] Although higher rates of unpublished preprints and publication retraction during this time support that view, 26 higher readership and scrutiny could also drive these retractions. 27,28 Thus, the unique circumstances and characteristics of COVID-19 publications call for careful evaluation of the quality and validity of the resulting literature, with an eye toward its evolution over time.…”

Bibliometric analysis of the 100 most-cited dental articles on COVID-19

“…Furthermore, there was a weak but significant association between citation counts and journal impact factors (r=0.241) (p<0.05). Gai et al 15 also reported that the pandemic-related articles have different impact metrics and article characteristics as compared to non-COVID studies. When research hot topics of the T100 articles were analyzed, as expected, the initial studies mainly focus on the epidemic transmission dynamics, prevention, and control measures of COVID-19 during dental procedures.…”