Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice

Abstract

Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines ( N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500–1200 words) and time-to-submit (median 12, IQR 4–26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400–550 words) and time-to-submit (median 4, IQR 4–6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

Methodological Rigor and Quality of Reporting of Clinical Trials Published with Physical Activity Interventions: A Report from the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative)

“…However, a collective effort is crucial to improve the scienti c culture. In this regard, we underscore the limited engagement of authors and journal editors in accessing evaluations, and similar issues of author receptivity to criticism have been noted in other studies (29,30).…”

Application of AI in Sepsis: Citation Network Analysis and Evidence Synthesis

“…Authors take various factors into account when submitting, such as the accessibility of open-access journals and the higher impact factor associated with certain subscription journals [ 33 ]. Some authors prefer open-access journals for their widespread availability after publication and the potential for a higher number of citations [ 34 ]. Conversely, subscription journals from reputable publishers attract attention due to their high impact factors [ 35 , 36 ].…”

Inaccuracy in the Scientific Record and Open Postpublication Critique

“…Examples of postpublication peer review are readily available in a range of online forums and blogs concerned with making science more open and accountable. A more specific mechanism is postpublication critique, defined by Hardwicke et al (2022 , Supplementary Information K) as “any journal-based avenue for sharing peer-initiated critical discourse related to specific research articles previously published in the same journal.” With the occasional exception ( Harms & Crede, 2020 ), such remedial strategies have not been discussed in mainstream psychology journals.…”