Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation

Casnici, Niccolò; Grimaldo, Francisco; Gilbert, Nigel; Dondio, Pierpaolo; Squazzoni, Flaminio

doi:10.1007/s11192-017-2241-1

Cited by 23 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The monitoring of its effectiveness and its influence upon research productivity has been the subject of great interest (Horlesberger et al 2013 ; Bruce et al 2016 ). Its objectiveness has been debated by many, as has its effect on scientific performance (Casnici et al 2017 ). It also may be the subject of inconsistencies that are difficult to remediate (Haug 2015 ).…”

Section: Introductionmentioning

confidence: 99%

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

et al. 2018

View full text Add to dashboard Cite

The Momentum program launched in 2009 provides funding of up to 1 million Euro to establish new, independent research groups at Hungarian academic institutions. Here, our aim was to determine factors associated with the scientific output of these research groups. Publication data were downloaded from the Hungarian Scientific Work Archive (www.mtmt.hu), impact factor data were obtained from Thomson Reuters (jcr.incites.thomsonreuters.com), and journal ranks were extracted from the Scimago Journal Rank database (www.scimagojr.com). Investigated input features for each grant holder include gender, degree, targeted category, international mobility, international grants, number of publications, total number of citations, H-index, best publications, impact factors in the last 2 years, and assessment scores provided by the experts. Evaluated performance indicators include cumulative impact factor, number of D1 publications, and number of first/last author D1 publications during the grant running time. Grant holders’ publication output increased by 23 and 52% for life sciences and material sciences researchers. Scientific performance was independent from gender, degree, international grants, category applied for, and citations received for the best pre-grant publication. Those with international mobility had significantly lower scientific output (yearly impact factor, number of D1 publications, number of first/last author publications). Scores received from grant review experts were independent from later publication activity. The strongest correlations were observed between scientific output and total number of citations, H-index, and impact factor in the last 2 years pre-grant. In summary, group leaders with a dynamic publication track record were able to attain the most additional momentum. Our results can help accelerate and improve future grant review processes.

show abstract

Section: Introductionmentioning

confidence: 99%

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Aiming, with substantial European research funding, to 'improve [the] efficiency, transparency and accountability of peer review through a trans-disciplinary, cross-sectorial collaboration', the consortium has been one of the most prolific centres for research into peer review in the past half decade. Publications from the group have spanned the author perspective on peer review (Drvenica et al 2019), the reward systems of peer review (Zaharie and Seeber 2018), the links between reputation and peer review (Grimaldo, Paolucci, and Sabater-Mir 2018), the role that artificial intelligence might play in future structures of review (Mrowinski et al 2017), the timescales involved in review (Huisman and Smits 2017;Mrowinski et al 2016), the reasons why people cite retracted papers (Bar-Ilan and Halevi 2017), the fate of rejected manuscripts (Casnici, Grimaldo, Gilbert, Dondio et al 2017), and the ways in which referees act in multidisciplinary contexts .…”

Section: The Study Of Peer Reviewmentioning

confidence: 99%

Reading Peer Review

Eve

Neylon

O’Donnell

et al. 2020

View full text Add to dashboard Cite

This Element describes for the first time the database of peer review reports at PLOS ONE, the largest scientific journal in the world, to which the authors had unique access. Specifically, this Element presents the background contexts and histories of peer review, the data-handling sensitivities of this type of research, the typical properties of reports in the journal to which the authors had access, a taxonomy of the reports, and their sentiment arcs. This unique work thereby yields a compelling and unprecedented set of insights into the evolving state of peer review in the twenty-first century, at a crucial political moment for the transformation of science. It also, though, presents a study in radicalism and the ways in which PLOS's vision for science can be said to have effected change in the ultra-conservative contemporary university. This title is also available as Open Access on Cambridge Core.

show abstract

“…Other important contributions that are more recent also argue for the importance of reviewing as a means of improving quality. Casnici et al (2017) note the importance of the number of times a paper is reviewed as a possible cause of the higher citation rates of papers. They studied papers which had been rejected by The Journal of Artificial Societies and Social Simulation (JASSS) but which were subsequently published elsewhere.…”

Section: Literaturementioning

confidence: 99%

Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation

2018

View full text Add to dashboard Cite

Journal peer review lies at the heart of academic quality control. This article explores the journal peer review process and seeks to examine how the reviewing process might itself contribute to papers, leading them to be more highly cited and to achieve greater recognition. Our work builds on previous observations and views expressed in the literature about (a) the role of actors involved in the research and publication process that suggest that peer review is inherent in the research process and (b) on the contribution reviewers themselves might make to the content and increased citation of papers. Using data from the journal peer review process of a single journal in the Social Sciences field (Business, Management and Accounting), we examine the effects of peer review on papers submitted to that journal including the effect upon citation, a novel step in the study of the outcome of peer review. Our detailed analysis suggests, contrary to initial assumptions, that it is not the time taken to revise papers but the actual number of revisions that leads to greater recognition for papers in terms of citation impact. Our study provides evidence, albeit limited to the case of a single journal, that the peer review process may constitute a form of knowledge production and is not the simple correction of errors contained in submitted papers.

show abstract

Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation

Cited by 23 publications

References 36 publications

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

Reading Peer Review

Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation

Contact Info

Product

Resources

About