A code reviewer assignment model incorporating the competence differences and participant preferences

Abstract

Abstract.A good assignment of code reviewers can effectively utilize the intellectual resources, assure code quality and improve programmers' skills in software development. However, little research on reviewer assignment of code review has been found. In this study, a code reviewer assignment model is created based on participants' preference to reviewing assignment. With a constraint of the smallest size of a review group, the model is optimized to maximize review outcomes and avoid the negative impact of "mutual admiration society". This study shows that the reviewer assignment strategies incorporating either the reviewers' preferences or the authors' preferences get much improvement than a random assignment. The strategy incorporating authors' preference makes higher improvement than that incorporating reviewers' preference. However, when the reviewers' and authors' preference matrixes are merged, the improvement becomes moderate. The study indicates that the majority of the participants have a strong wish to work with reviewers and authors having highest competence. If we want to satisfy the preference of both reviewers and authors at the same time, the overall improvement of learning outcomes may be not the best.

Modern Code Reviews—Survey of Literature and Practice

“…In one study, the authors used participants' preferences in review assignment [263], while in the other study, the authors combined the metadata of pull requests with the metadata associated with potential reviewers [92]. Another study focuses on detecting and removing systematic labeling bias to improve prediction [235].…”

A Survey on Online Judge Systems and Their Applications

“…In turn, Cruz et al presented how to extend the typical online judge architecture to develop a system that goes a step further by providing valuable feedback to users at a semantic level in the form of meaningful advice to understand where the problem is and how to improve the code (Fonte et al 2013). Such an approach can significantly impact the student learning process (Wang et al 2016). Finally, there is an interesting idea of building computer games that integrate or wrap online judge systems to be more attractive to users (Ivanova 2016).…”