An algorithm to determine peer-reviewers

Abstract

The peer-review process is the most widely accepted certification mechanism for officially accepting the written results of researchers within the scientific community. An essential component of peer-review is the identification of competent referees to review a submitted manuscript. This article presents an algorithm to automatically determine the most appropriate reviewers for a manuscript by way of a co-authorship network data structure and a relative-rank particle-swarm algorithm. This approach is novel in that it is not limited to a pre-selected set of referees, is computationally efficient, requires no human-intervention, and, in some instances, can automatically identify conflict of interest situations. A useful application of this algorithm would be to open commentary peer-review systems because it provides a weighting for each referee with respects to their expertise in the domain of a manuscript. The algorithm is validated using referee bid data from the 2005 Joint Conference on Digital Libraries.

A robust model for paper reviewer assignment

“…Then they defined some specific matching criteria to optimize the reviewer arrangement procedure. Rodriguez et al [9] built a co-authorship graph with the references of a submitted paper as starting points to suggest reviewers. Conry et al [6] first studied the preference of reviewers for specific papers as available feedbacks.…”

A Taxonomy-Based Model for Expertise Extrapolation

“…Table III summarizes topic contributions from the various data sources before curation. We note that many attempts that led to the development of taxonomies across a variety of areas have been undertaken [8], [18], however, to date no gold standard taxonomy exists for computer science topics. Hence it is difficult to independently evaluate the quality of the taxonomy we created with other known sources, beyond with what has been attempted here.…”

“…In this work we exploit publicly available datasets for finding experts, we have demonstrated proof of concept such that scalability is possible in theory given the appropriate resources. In [8] various algorithms for finding relevant reviewers are presented, based on coauthorship graphs and relative-rank particle-swarm propagation between coauthors. Edge weights propagate through the coauthorship network using stochastic analysis on outgoing edges, and state and energy levels of propagating nodes allow identification of the most qualified reviewers.…”

“…In the past a variety of sources have been used for finding experts in scientific research, including Citation linkage [3], Curricula Vitarum (CVs) [4], Intranet applications [5], [6], Version Control Systems (such as SVNs) [7], [8], patents and even research grants. However, the unavailability of such high quality datasets as SVNs, patents and intranet records, make them unsuitable for large scale tasks.…”

Discovery and visualization of expertise in a scientific community

“…Expertise finder systems in the past, have been innovatively applied in helping PhD applicants in finding relevant supervisors [15] and also in identifying peer-reviewers for a conference [20]. The former made use of a manually constructed expertise profile database while the latter employed reference mining for all papers submitted to a conference.…”