Publish or Politic: Referee Bias in Manuscript Review<sup>1</sup>

Abramowitz, Stephen I.; Gomes, Beverly; Abramowitz, Christine V.

doi:10.1111/j.1559-1816.1975.tb00675.x

Cited by 128 publications

(66 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Reviewers' recommendations on whether to publish were mostly consistent with their methodology ratings. Similar experimental findings were obtained in psychology by (Smart (1964), Goodstein and Brazis (1970), Abramowitz, Gomes, and Abramowitz (1975), and Koehler (1993) reported, and in biomedical research by Young, Ioannidis, and Al-Ubaydli (2008).…”

Section: Defining Useful Sciencesupporting

confidence: 76%

Guidelines for Science: Evidence and Checklists

Armstrong

Green

2017

SSRN Journal

View full text Add to dashboard Cite

Problem: The scientific method is unrivalled as a basis for generating useful knowledge, yet research papers published in management, economics, and other social sciences fields often ignore scientific principles. What, then, can be done to increase the publication of useful scientific papers?Methods: Evidence on researchers' compliance with scientific principles was examined. Guidelines aimed at reducing violations were then derived from established definitions of the scientific method. Findings:Violations of the principles of science are encouraged by: (a) funding for advocacy research; (b) regulations that limit what research is permitted, how it must be designed, and what must be reported; (c) political suppression of scientists' speech; (d) universities' use of invalid criteria to evaluate research-such as grant money and counting of publications without regard to usefulness; (e) journals' use of invalid criteria for deciding which papers to publish-such as the use of statistical significance tests. Solutions:We created a checklist of 24 evidence-based operational guidelines to help researchers comply with scientific principles (valid inputs). Based on the definition of science, we then developed a checklist of seven criteria to evaluate whether a research paper provides useful scientific findings (valuable outputs). That checklist can be used by researchers, funders, courts, legislators, regulators, employers, reviewers, and journals.Originality: This paper provides the first comprehensive evidence-based checklists of operational guidelines for conducting scientific research and for evaluating the scientific quality and usefulness of research efforts.Usefulness: Journals could increase the publication of useful papers by including a section committed to publishing all relevant and useful papers that comply with science. By using the Criteria for Useful Science checklist, those who support science could more effectively evaluate the contributions of scientists. AbstractProblem: The scientific method is unrivalled as a basis for generating useful knowledge, yet research papers published in management, economics, and other social sciences fields often ignore scientific principles. What, then, can be done to increase the publication of useful scientific papers?Methods: Evidence on researchers' compliance with scientific principles was examined. Guidelines aimed at reducing violations were then derived from established definitions of the scientific method.Findings: Violations of the principles of science are encouraged by: (a) funding for advocacy research; (b) regulations that limit what research is permitted, how it must be designed, and what must be reported; (c) political suppression of scientists' speech; (d) universities' use of invalid criteria to evaluate research-such as grant money and counting of publications without regard to usefulness; (e) journals' use of invalid criteria for deciding which papers to publish-such as the use of statistical significance tests.Solutions: We created a chec...

show abstract

Section: Defining Useful Sciencesupporting

confidence: 76%

Guidelines for Science: Evidence and Checklists

Armstrong

Green

2017

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…They rated those in which the results were in accord with their own beliefs as better designed and said that they were more suitable for publication. Abramowitz, Gomes, and Abramowitz [1975] did a similar study and reached the same conclusion.…”

Section: Research On Management Folkloresupporting

confidence: 71%

The Ombudsman: Management Folklore and Management Science—On Portfolio Planning, Escalation Bias, and Such

Armstrong

1996

Interfaces

View full text Add to dashboard Cite

Management folklore sometimes leads to unprofitable decision making. Thus, studies of the value of such folklore should be of interest to managers, especially when they identify unprofitable procedures. I reviewed empirical research on scientific publishing and concluded that studies supporting management folklore are likely to be favorably reviewed for publication and to be cited. However, researchers who obtain findings that refute folklore are likely to encounter resistance in publication and are less likely to be cited. My experience with papers on portfolio planning methods and escalation bias illustrates the problem. To encourage the publication of papers that challenge management folklore, editors should use results-blind reviews and, in some cases,constrain, reduce, or eliminate peer review. AbstractManagement folklore sometimes leads to unprofitable decision making. Thus, studies of the value of such folklore should be of interest to managers, especially when they identify unprofitable procedures. I reviewed empirical research on scientific publishing and concluded that studies supporting management folklore are likely to be favorably reviewed for publication and to be cited. However, researchers who obtain findings that refute folklore are likely to encounter resistance in publication and are less likely to be cited. My experience with papers on portfolio planning methods and escalation bias illustrates the problem. To encourage the publication of papers that challenge management folklore, editors should use results-blind reviews and, in some cases, constrain, reduce, or eliminate peer review.

show abstract

“…They are probably not even aware of such a prejudice. However, experimental studies (Goodstein and Brazis, 1970;Abramovitz et al, 1975;Mahoney, 1977) show that such a bias does exist.…”

Section: Effects Of Peer Review On the Publication Of Controversial Fmentioning

confidence: 98%

Publication of research on controversial topics: The early acceptance procedure

Armstrong

1996

International Journal of Forecasting

View full text Add to dashboard Cite

Publish or Politic: Referee Bias in Manuscript Review¹

Cited by 128 publications

References 22 publications

Guidelines for Science: Evidence and Checklists

Guidelines for Science: Evidence and Checklists

The Ombudsman: Management Folklore and Management Science—On Portfolio Planning, Escalation Bias, and Such

Publication of research on controversial topics: The early acceptance procedure

Contact Info

Product

Resources

About

Publish or Politic: Referee Bias in Manuscript Review1

Cited by 128 publications

References 22 publications

Guidelines for Science: Evidence and Checklists

Guidelines for Science: Evidence and Checklists

The Ombudsman: Management Folklore and Management Science—On Portfolio Planning, Escalation Bias, and Such

Publication of research on controversial topics: The early acceptance procedure

Contact Info

Product

Resources

About

Publish or Politic: Referee Bias in Manuscript Review¹