Improving code review effectiveness through reviewer recommendations

Thongtanunam, Patanamon; Kula, Raula Gaikovina; Cruz, Ana Erika Camargo; Yoshida, N.; Iida, Hajimu

doi:10.1145/2593702.2593705

Cited by 52 publications

(36 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The common idea behind these approaches is to automatically identify potential reviewers who are the most suitable for a given change. The main proxy for suitability estimation is expertise (or familiarity) of candidates with code under review, which is estimated through analysis of artifacts of developers' prior work, such as histories of code changes and review participation [4], [12], [56].…”

Section: Reviewer Recommendationmentioning

confidence: 99%

See 1 more Smart Citation

Does Reviewer Recommendation Help Developers?

Kovalenko

Tintarev

Pasynkov³

et al. 2020

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Abstract-Selecting reviewers for code changes is a critical step for an efficient code review process. Recent studies propose automated reviewer recommendation algorithms to support developers in this task. However, the evaluation of recommendation algorithms, when done apart from their target systems and users (i.e., code review tools and change authors), leaves out important aspects: perception of recommendations, influence of recommendations on human choices, and their effect on user experience. This study is the first to evaluate a reviewer recommender in vivo. We compare historical reviewers and recommendations for over 21,000 code reviews performed with a deployed recommender in a company environment and set out to measure the influence of recommendations on users' choices, along with other performance metrics. Having found no evidence of influence, we turn to the users of the recommender. Through interviews and a survey we find that, though perceived as relevant, reviewer recommendations rarely provide additional value for the respondents. We confirm this finding with a larger study at another company. The confirmation of this finding brings up a case for more user-centric approaches to designing and evaluating the recommenders. Finally, we investigate information needs of developers during reviewer selection and discuss promising directions for the next generation of reviewer recommendation tools. Preprint: https://doi.org/10.5281/zenodo.1404814

show abstract

Section: Reviewer Recommendationmentioning

confidence: 99%

“…Some techniques are based on scoring of candidates, either based on changes history at line level [12] or on analysis of historical reviewers for files with similar paths [56]. Another approach is machine learning on change features [57].…”

Section: Reviewer Recommendationmentioning

confidence: 99%

Does Reviewer Recommendation Help Developers?

Kovalenko

Tintarev

Pasynkov³

et al. 2020

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…isCollect(ActualReviewers r , TopN) has a value of 1 if TopN includes at least one reviewer involved in ActualReviewer r ; otherwise, it has a value of 0. According to [6,10,33], we chose the value of N to be 1, 3, 5, and 10.…”

Section: Evaluation Metricsmentioning

confidence: 99%

Understanding Review Expertise of Developers: A Reviewer Recommendation Approach Based on Latent Dirichlet Allocation

Kim

Lee

2018

Symmetry

View full text Add to dashboard Cite

Abstract:The code reviewer assignment problem affects the reviewing time of a source code change. To effectively perform the code review process of a software project, the code reviewer assignment problem must be dealt with. Reviewer recommendation can reduce the time required for finding appropriate reviewers for a given source code change. In this paper, we propose a reviewer recommendation approach based on latent Dirichlet allocation (LDA). The proposed reviewer recommendation approach consists of a review expertise generation phase and a reviewer recommendation phase. The review expertise generation phase generates the review expertise of developers for topics of source code changes from the review history of a software project. The reviewer recommendation phase computes the review scores of the developers according to the topic distribution of a given source code change and the review expertise of the developers. In an empirical evaluation of five open source projects, we confirm that the proposed reviewer recommendation approach obtains better average top-10 accuracy than existing reviewer recommendation approaches.

show abstract

“…The exact number of activities in each day will be shown in the description box at the top right when the cursor is hovering over the graph 3(a). ReDA also shows the date for Android release version 5 via a dashed line to show how these activities are before/after the release date. For example, as shown in Fig.…”

Section: B Activity Statisticmentioning

confidence: 99%

“…Extracting knowledge from these datasets has produced promising research with the goal of improving the software quality and software development process. Recently, many studies have used code review datasets to understand and improve both review effort [2]- [5] and review quality [6], [7]. However, a raw code review dataset is generally imperfect since the data collection process in each support tool 1 https://code.google.com/p/gerrit/ 2 https://www.reviewboard.org/ 3 https://code.google.com/p/rietveld/ varies in methodology, accuracy, and degree of automation [8].…”

Section: Introductionmentioning

confidence: 99%

ReDA: A Web-Based Visualization Tool for Analyzing Modern Code Review Dataset

Thongtanunam

Yang

Yoshida

et al. 2014

2014 IEEE International Conference on Software Maintenance and Evolution

Self Cite

View full text Add to dashboard Cite

Abstract-ReDA(http://reda.naist.jp/) is a web-based visualization tool for analyzing Modern Code Review (MCR) datasets for large Open Source Software (OSS) projects. MCR is a commonly practiced and lightweight inspection of source code using a support tool such as Gerrit system. Recently, mining code review history of such systems has received attention as a potentially effective method of ensuring software quality. However, due to increasing size and complexity of softwares being developed, these datasets are becoming unmanageable. ReDA aims to assist researchers of mining code review data by enabling better understand of dataset context and identifying abnormalities. Through real-time data interaction, users can quickly gain insight into the data and hone in on interesting areas to investigate. A video highlighting the main features can be found at: http://youtu.be/ fEoTRRas0U

show abstract

Improving code review effectiveness through reviewer recommendations

Cited by 52 publications

References 13 publications

Does Reviewer Recommendation Help Developers?

Does Reviewer Recommendation Help Developers?

Understanding Review Expertise of Developers: A Reviewer Recommendation Approach Based on Latent Dirichlet Allocation

ReDA: A Web-Based Visualization Tool for Analyzing Modern Code Review Dataset

Contact Info

Product

Resources

About