Computational support for academic peer review

Abstract

New tools tackle an age-old practice. BY SIMON PRICE AND PETER A. FLACH key insights ˽ State-of-the-art tools from machine learning and artificial intelligence are making inroads to automate parts of the peer-review process; however, many opportunities for further improvement remain.

Perspectives for early-career researchers on plagiarism and scientific integrity

“…Awareness of these biases is increasing, and prominent journals are trying to address them. The review process will continue to evolve, with some advocating increased use of plagiarism-matching software while others suggest the use of artificial intelligence in place of firstpass reviewers (Price and Flach 2017). Understanding the complexities of review, often gained through participation in the process as a reviewer, can help ECRs increase their success at publishing.…”

Challenges with Reproducibility

“…Good reviews take time and the community usually has a limited pool of people providing good (substantial and constructive) reviews. Matching reviewers with submi ed papers is also becoming a challenge, to such a degreee that conferences are now experimenting [12,17] with automated review assignment systems. Checking for reproducibility increases review expectations even further thereby shrinking the pool of good reviewers.…”

Electronic Markets on academic supply chains

“…The renaissance of artificial intelligence (AI) could add functionalities to review systems that address the allocation of reviewers to submissions. A recent contribution by Price and Flach (2017) conceived this decision as combinatorial allocation problem that may be solved via score matrixes based on available techniques from information retrieval and machine learning. The rationale is not only to reduce workload for responsible editors, but also to augment the probability that invited reviewers are more likely to agree if a paper falls within their area of expertise.…”