A robust model for paper reviewer assignment

Liu, Xiang; Suel, Torsten; Memon, Nasir

doi:10.1145/2645710.2645749

Cited by 65 publications

(63 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bootstrapping can be done using traditional similarity measures. Liu et al (2014) proposes a model using Random Walk with Restart to traverse the graph of potential reviewers. The information on authority, expertise and diversity are used to build this network.…”

Section: Reviewer Assignment Problemmentioning

confidence: 99%

Profile based recommendation of code reviewers

2017

View full text Add to dashboard Cite

Code reviews consist in proof-reading proposed code changes in order to find their shortcomings such as bugs, insufficient test coverage or misused design patterns. Code reviews are conducted before merging submitted changes into the main development branch. The selection of suitable reviewers is crucial to obtain the high quality of reviews. In this article we present a new method of recommending reviewers for code changes. This method is based on profiles of individual programmers. For each developer we maintain his/her profile. It is the multiset of all file path segments from commits reviewed by him/her. It will get updated when he/she presents a new review. We employ a similarity function between such profiles and change proposals to be reviewed. The programmer whose profile matches the change most is recommended to become the reviewer. We performed an experimental comparison of our method against state-of-the-art techniques using four large open-source projects. We obtained improved results in terms of classification metrics (precision, recall and F-measure) and performance (we have lower time and space complexity).

show abstract

Section: Reviewer Assignment Problemmentioning

confidence: 99%

Profile based recommendation of code reviewers

2017

View full text Add to dashboard Cite

show abstract

“…Firstly, the retrieval-based approaches, such as latent semantic index (LSI) [39], topic model [4,31,36], etc., mainly focus on the topic relationship between papers and reviewer candidates. Besides, these methods have considered many types of constraints, such as authority, diversity, expertise, availability [18,24,26,41] and the research interests [17] of reviewers. Secondly, the matching-based approaches often compute a matching based on a bipartite graph between papers and reviewer candidates [5,6].…”

Section: Introductionmentioning

confidence: 99%

A Multi-Label Classification Method Using a Hierarchical and Transparent Representation for Paper-Reviewer Recommendation

Zhang

Zhao

Duan

et al. 2020

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Paper-reviewer recommendation task is of significant academic importance for conference chairs and journal editors. It aims to recommend appropriate experts in a discipline to comment on the quality of papers of others in that discipline. How to effectively and accurately recommend reviewers for the submitted papers is a meaningful and still tough task. Generally, the relationship between a paper and a reviewer often depends on the semantic expressions of them. Creating a more expressive representation can make the peer-review process more robust and less arbitrary. So, the representations of a paper and a reviewer are very important for the paper-reviewer recommendation. Actually, a reviewer or a paper often belongs to multiple research fields, which increases difficulty in paper-reviewer recommendation. In this paper, we propose a Multi-Label Classification method using a HIErarchical and transPArent Representation named Hiepar-MLC. Firstly, we introduce a HIErarchical and transPArent Representation (Hiepar) to express the semantic information of the reviewer and the paper. Hiepar is learned from a two-level bidirectional gated recurrent unit based network applying the attention mechanism. It is capable of capturing the two-level hierarchical information (word-sentence-document) and highlighting the elements in reviewers or papers to support the labels. This word-sentence-document information mirrors the hierarchical structure of a reviewer or a paper and captures the exact semantics of them. Then we transform the paper-reviewer recommendation problem into a Multi-Label Classification (MLC) issue, whose multiple research labels exactly guide the learning process. It's flexible that we can select any multi-label classification method to solve the paper-reviewer recommendation problem. Further, we propose a simple multi-label based reviewer assignment (MLBRA) strategy to select the appropriate reviewers. It's interesting that we also explore the paper-reviewer recommendation in the coarse-grained granularity. Extensive experiments on the real-world dataset consisting of the papers in the ACM Digital Library show that MLC-Hiepar achieves better label prediction performance than the existing representation alternatives. Also, with the MLBRA strategy, we show the effectiveness and the feasibility of our transformation from paper-reviewer recommendation to multi-label classification. . 2019. A multi-label classification method using a hierarchical and transparent representation for paper-reviewer recommendation.

show abstract

“…Without real-time matching, for example, if one waits to match people to teams offline (e.g., by collecting a pool of people before assigning teams), then team starvation or worker retention issues arise where teams may sit dormant without progress or workers may leave while you wait to assign people to the teams. The problem of real-time diverse matching arises in many disciplines and problems, including: matching workers to firms [5], children to schools [6,7], reviewers to manuscripts [8,9], donor organs to patients [10,11] and residents to public housing [12]. Specifically, this paper contributes to the following:…”

Section: Introductionmentioning

confidence: 99%

Forming Diverse Teams From Sequentially Arriving People

Ahmed

Dickerson

Fuge

2020

Journal of Mechanical Design

View full text Add to dashboard Cite

Collaborative work often benefits from having teams or organizations with heterogeneous members. In this paper, we present a method to form such diverse teams from people arriving sequentially over time. We define a monotone submodular objective function that combines the diversity and quality of a team and propose an algorithm to maximize the objective while satisfying multiple constraints. This allows us to balance both how diverse the team is and how well it can perform the task at hand. Using crowd experiments, we show that, in practice, the algorithm leads to large gains in team diversity. Using simulations, we show how to quantify the additional cost of forming diverse teams and how to address the problem of simultaneously maximizing diversity for several attributes (e.g. country of origin, gender). Our method has applications in collaborative work ranging from team formation, the assignment of workers to teams in crowdsourcing, and reviewer allocation to journal papers arriving sequentially. Our code is publicly accessible for further research.

show abstract

A robust model for paper reviewer assignment

Cited by 65 publications

References 20 publications

Profile based recommendation of code reviewers

Profile based recommendation of code reviewers

A Multi-Label Classification Method Using a Hierarchical and Transparent Representation for Paper-Reviewer Recommendation

Forming Diverse Teams From Sequentially Arriving People

Contact Info

Product

Resources

About