The evolution of peer review as a basis for scientific publication: directional selection towards a robust discipline?

Ferreira, Catarina; Bastille‐Rousseau, Guillaume; Bennett, Amanda M.; Ellington, E. Hance; Terwissen, C. V.; Austin, Cayla; Borlestean, Adrian; Boudreau, Melanie R.; Chan, K. Y.; Forsythe, Adrian; Hossie, Thomas J.; Landolt, Kristen N.; Longhi, Jessica N.; Otis, Josée-Anne; Peers, Michael J. L.; Rae, Jason; Seguin, Jacob L.; Watt, Cristen; Wehtje, Morgan; Murray, Dennis L.

doi:10.1111/brv.12185

Cited by 21 publications

(18 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…that the journal editor's tendency to match submissions with a diverse set of referees, for both disciplinary background and seniority, was instrumental in evaluating multidisciplinary submissions fairly. This would confirm the importance of carefully selecting referees in peer review and ensuring a diversity of criteria of judgment and opinions (e.g., Ferreira et al, 2015).…”

Section: Discussionmentioning

confidence: 76%

Attitudes of referees in a multidisciplinary journal: An empirical analysis

Casnici

Grimaldo

Gilbert

et al. 2016

Asso for Info Science & Tech

View full text Add to dashboard Cite

This paper looks at 10 years of reviews in a multidisciplinary journal, The Journal of Artificial Societies and Social Simulation (JASSS), which is the flagship journal of social simulation. We measured referee behavior and referees' agreement. We found that the disciplinary background and the academic status of the referee have an influence on the report time, the type of recommendation and the acceptance of the reviewing task. Referees from the humanities tend to be more generous in their recommendations than other referees, especially economists and environmental scientists. Second, we found that senior researchers are harsher in their judgments than junior researchers, and the latter accept requests to review more often and are faster in reporting. Finally, we found that articles that had been refereed and recommended for publication by a multidisciplinary set of referees were subsequently more likely to receive citations than those that had been reviewed by referees from the same discipline. Our results show that common standards of evaluation can be established even in multidisciplinary communities. IntroductionReviewing for journals is a kind of moral duty in the scientific community, being instrumental to the Mertonian ethical norms that regulate science as an organized institution Merton 1942). We know that the quality of our publications depends at least partially on comments and suggestions given by competent and cooperative referees (e.g., Mulligan, Hall, & Raphael, 2013). On the other hand, we know that science is a public good that can be maintained only if we are unbiased in judgment and collaborate in distributing efficiently and more or less equally the reviewing effort (e.g., Hochberg, Chase, Gotelli, Hastings, & Naeem, 2009).Given that review standards are not formalized and our decisions are typically confidential, it is likely that the way we accomplish this duty may depend on our background and experience, as well as on our commitment to the journal that asked our opinion. Given the lack of training on reviewing, the opacity of the process, and the weak incentives for referees, the way we review and the time we take to accomplish this important task might depend on attitudes and norms that can reflect the attitudes of the other members of our scientific community (e.g., Azar, 2008;Squazzoni & Gandelli, 2013).This means that looking at referee behavior could potentially help to reveal scientist misbehavior or situations where referees could benefit from their gatekeeping role at the This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.expense of editors and/or authors (e.g., Bornmann, Weymuth, & Daniel, 2010;Lamont, 2009;Garc ıa, Rodriguez-S anchez & Fdez-Valdivia, 2015); it could also inform us about the nature of the social norms of reviewing and so help to counter...

show abstract

Section: Discussionmentioning

confidence: 76%

Attitudes of referees in a multidisciplinary journal: An empirical analysis

Casnici

Grimaldo

Gilbert

et al. 2016

Asso for Info Science & Tech

View full text Add to dashboard Cite

show abstract

“…Their evaluation is covered by an editorial peer review process (see for example Ferreira et al (2016) for history and future of peer review), and assessing the merit of an argument is beyond the scope of this work.…”

Section: Methodsmentioning

confidence: 99%

Reproducible research and GIScience: an evaluation using AGILE conference papers

Nüst

Granell

Hofer

et al. 2018

PeerJ

View full text Add to dashboard Cite

The demand for reproducible research is on the rise in disciplines concerned with data analysis and computational methods. Therefore, we reviewed current recommendations for reproducible research and translated them into criteria for assessing the reproducibility of articles in the field of geographic information science (GIScience). Using this criteria, we assessed a sample of GIScience studies from the Association of Geographic Information Laboratories in Europe (AGILE) conference series, and we collected feedback about the assessment from the study authors. Results from the author feedback indicate that although authors support the concept of performing reproducible research, the incentives for doing this in practice are too small. Therefore, we propose concrete actions for individual researchers and the GIScience conference series to improve transparency and reproducibility. For example, to support researchers in producing reproducible work, the GIScience conference series could offer awards and paper badges, provide author guidelines for computational research, and publish articles in Open Access formats.

show abstract

“…Simultaneously, science has become more competitive, and, thus, the new technology was also used to introduce journal and researcher citation metrics as bases for the competition between journals, institutions, and researchers. Within this system, the journal editor's position has shifted toward that of a promoter who wants to ensure the journal's success by including articles that have the greatest chance of being cited quickly and often …”

Section: The Social Sciences Perspective On the Use Of Information Tementioning

confidence: 99%

“…Within this system, the journal editor's position has shifted toward that of a promoter who wants to ensure the journal's success by including articles that have the greatest chance of being cited quickly and often. 33 The social sciences' perspective that the use of tools to enhance our thinking leads to satisfaction with work and influences social relations serves as a framework to explain the problems health professionals have had using health IT. The three functional cognitive complexes that are enhanced by the use of IT tools can be used to structure an explanation and to derive ideas for improvement: (1) using health IT for documenting patient-related information, (2) using health IT for communicating patientrelated information at the point of care, or (3) using health IT for clinical decision support.…”

Section: The Social Sciences Perspective On the Use Of Information Tementioning

confidence: 99%

Information technology and social sciences: how can health IT be used to support the health professional?

Wagner‐Menghin

Pokieser

2016

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Keeping up to date with the increasing amount of health-related knowledge and managing the increasing numbers of patients with more complex clinical problems is a challenge for healthcare professionals and healthcare systems. Health IT applications, such as electronic health records or decision-support systems, are meant to support both professionals and their support systems. However, for physicians using these applications, the applications often cause new problems, such as the impracticality of their use in clinical practice. This review adopts a social sciences perspective to understand these problems and derive suggestions for further development. Indeed, humans use tools to remediate the brain's weaknesses and enhance thinking. Available health IT tools have been shaped to fit administrative needs rather than physicians' needs. To increase the beneficial effect of health IT applications in health care, clinicians' style of thinking and their learning needs must be considered when designing and implementing such systems. New health IT tools must be shaped to fit health professionals' needs. To further ease the integration of new health IT tools into clinical practice, we must also consider the effects of implementing new tools on the wider social framework.

show abstract

The evolution of peer review as a basis for scientific publication: directional selection towards a robust discipline?

Cited by 21 publications

References 69 publications

Attitudes of referees in a multidisciplinary journal: An empirical analysis

Attitudes of referees in a multidisciplinary journal: An empirical analysis

Reproducible research and GIScience: an evaluation using AGILE conference papers

Information technology and social sciences: how can health IT be used to support the health professional?

Contact Info

Product

Resources

About