An Eye-Tracking Study on the Use of Split/Unified Code Change Views for Bug Detection

Abstract

To address a bug detection task, developers often compare two versions of source code. Comparison tools support this task by showing both versions of the code in a split view or unified way. However, there is still little knowledge of how developers interact with code comparison tools during a source code change analysis task. Therefore, the strengths and weaknesses of each approach remain unknown. To address this research gap, we investigate their usefulness, limitations, and potential room for improvement. To this end, we conducted a user study involving 12 participants who analyzed the source code in two commits to detect bugs. In the study, we adopted a within-subjects approach. Each participant addressed similar tasks using the split and unified views of GitHub while using an eye-tracking device. We found that participants experienced less visual effort when using the unified view. The results suggest that a lower effort can promote a deeper analysis of the code, which often leads to the detection of more bugs. We also observed that participants predominantly looked at conditionals, class/instance variables, and code changes. However, we did not find statistically significant differences among the variables analyzed when using split or unified views. We consider the results of the study to be relevant to practitioners and researchers alike. Practitioners will be informed when choosing a view, while researchers will identify opportunities to improve code comparison tools.INDEX TERMS bug detection, code review, eye-tracking, user study