Using Machine Intelligence to Prioritise Code Review Requests

Abstract

Modern Code Review (MCR) is the process of reviewing new code changes that need to be merged with an existing codebase. As a developer, one may receive many code review requests every day, i.e., the review requests need to be prioritised. Manually prioritising review requests is a challenging and timeconsuming process. To address the above problem, we conducted an industrial case study at Ericsson aiming at developing a tool called Pineapple, which uses a Bayesian Network to prioritise code review requests. To validate our approach/tool, we deployed it in a live software development project at Ericsson, wherein more than 150 developers develop a telecommunication product. We focused on evaluating the predictive performance, feasibility, and usefulness of our approach. The results indicate that Pineapple has competent predictive performance (RMSE = 0.21 and MAE = 0.15). Furthermore, around 82.6% of Pineapple's users believe the tool can support code review request prioritisation by providing reliable results, and around 56.5% of the users believe it helps reducing code review lead time. As future work, we plan to evaluate Pineapple's predictive performance, usefulness, and feasibility through a longitudinal investigation.

Modern Code Reviews—Survey of Literature and Practice

“…Code reviewers often need to prioritize which changes they should focus on reviewing first. Many studies propose to base the review decision on the likelihood that a particular change will eventually be accepted/merged [53,105,213]. Fan et al [105] proposed an approach based on Random Forest.…”

“…They evaluated their approach using data from open source projects, and obtained solid results. Saini and Britto [213] developed a Bayesian Network to predict acceptance probability. The acceptance probability is combined with other aspects, such as task type and the presence of merge conflicts, to order the list of code review requests associated with a developer.…”

Evaluating Learning-to-Rank Models for Prioritizing Code Review Requests using Process Simulation

“…In fact, a good prioritization mechanism can help detect important and urgent code changes (e.g., fixes for high-impact bugs), which can be merged more quickly and their delivery to post-review software activities accelerated. Also, such a mechanism can help code reviewers better schedule their time and pay attention to the high-priority CRRs, and help code authors receive more timely feedback to improve their changes [14,15].…”

“…There is a wider range of complex considerations besides recency in practice, e.g., outcome and urgency [14]. Studies [14,15,16,17,18] have employed learning-based prioritizers to address this problem automatically. These prioritizers are based on a model (henceforth: Learning-to-Rank or LtR model) that decides on an optimal ordering of an entire list of CRRs.…”

“…Saini and Britto [15] constructed five LtR models to select the most appropriate one for constructing a CRR prioritizer, including Bayesian Network, Random Forests, Gradient Boosting, Logistic Regression, and k-Nearest Neighbors. Among them, Bayesian Network worked best on Root Mean Square Error and Mean Absolute Error metrics.…”