How to improve scientific peer review: Four schools of thought

Waltman, Ludo; Kaltenbrunner, Wolfgang; Pinfield, Stephen; Woods, Helen Buckley

doi:10.31235/osf.io/v8ghj

Cited by 7 publications

(11 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the number of variables involved in academic publishing, this is a multifaceted problem with different schools of thought on peer review improvement. Waltman et al (2022) argue that there are four different perspectives on how to improve peer review (focusing on Quality & Reproducibility, Democracy & Transparency, Equity & Inclusion, Efficiency & Incentives). These categories of schools of thought provide a useful framework for thinking about the implications of any changes to the review process.…”

Section: Meta-sciencementioning

confidence: 99%

Barriers and enabling factors for error analysis in NLG research

Miltenburg

Clinciu²,

Dušek³

et al. 2023

NEJLT

View full text Add to dashboard Cite

Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and discusses several ways to do so. This paper is not just based on our shared experience as authors, but we also distributed a survey as a means of public consultation. We provide an overview of existing barriers to carry out error analyses, and proposes changes to improve error reporting in the NLG literature.

show abstract

Section: Meta-sciencementioning

confidence: 99%

Barriers and enabling factors for error analysis in NLG research

Miltenburg

Clinciu²,

Dušek³

et al. 2023

NEJLT

View full text Add to dashboard Cite

show abstract

“…The peer review system is rightly touted as the best current system for producing independent, reliable scientific work, and it is for this reason that relying on peer-reviewed publications for informing policy is recommended. The current peer review system is, however, buckling for a multitude of reasons [8,9,10,11]. One reason for this is that peer-reviewing is currently voluntary and relies on indirect reciprocity-as such there is no incentive to dedicate the necessary time to complete the review thoroughly and rigorously, aside from a sense of moral obligation.…”

Section: Problem One: Lack Of Transparency Throughout the Research Pr...mentioning

confidence: 99%

An outdated publishing system threatens both research integrity and the retention of rigorous early career researchers

Brand

2022

Preprint

View full text Add to dashboard Cite

Research integrity and reproducibility are threatened by two key, intertwined problems in the current research system: 1) there is a lack of transparency throughout theresearch process and 2) scientific publishing relies on a faulty prestige economy maintained by current for-profit publishing companies. Both of these problems threaten not only the integrity and reproducibility of research outputs themselves, but also the retention and career progression of rigorous, dedicated early career research scientists within public research institutions. This submission reflects my personal experience as an early career researcher, and how I see these problems impacting other researchers who are also on temporary, precarious contracts, and seeking grant funding or permanent research positions. Please note a version of this article was originally written and published for the UK Parliament as part of their Call for Evidence on Research Integrity and Reproducibility in September 2021.

show abstract

“…In practice, however, different norms will give rise to contradicting measures (Waltman et al. 2022, 3). Our question is thus, should the forum or the de‐biasing through anonymisation camp be preferred by philosophers?…”

Section: Introduction: the Peer‐review Crisis In Philosophy And A Neg...mentioning

confidence: 99%

“It takes a village to write a really good paper”: A normative framework for peer reviewing in philosophy

Copeland,

Marin

2024

Metaphilosophy

View full text Add to dashboard Cite

That there is a “crisis of peer review” at the moment is not in dispute, but sufficient attention has not yet been paid to the normative potential that lies in current calls for reform. In contrast to approaches to “fixing” the problems in peer review, which tend to maintain the status quo in terms of professionalising opportunities, this paper addresses the needs of philosophers and how peer‐review reform can be an opportunity to improve the academic discipline of philosophy, whereby progress is understood as making the discipline more fair to the global academic community and more conducive to the flourishing of academic philosophers. The paper evaluates recent categories of relevant norms and correlating reforms. In conclusion, it recommends that philosophy pursue the norms of transparency and democracy explicitly when proposing peer‐review reform and suggest that proposals for forum‐based models of peer review are most likely to support those norms.

show abstract

How to improve scientific peer review: Four schools of thought

Cited by 7 publications

References 45 publications

Barriers and enabling factors for error analysis in NLG research

Barriers and enabling factors for error analysis in NLG research

An outdated publishing system threatens both research integrity and the retention of rigorous early career researchers

“It takes a village to write a really good paper”: A normative framework for peer reviewing in philosophy

Contact Info

Product

Resources

About