Developer Reading Behavior While Summarizing Java Methods: Size and Context Matters

Abstract

An eye-tracking study of 18 developers reading and summarizing Java methods is presented. The developers provide a written summary for methods assigned to them. In total, 63 methods are used from five different systems. Previous studies on this topic use only short methods presented in isolation usually as images. In contrast, this work presents the study in the Eclipse IDE allowing access to all the source code in the system. The developer can navigate via scrolling and switching files while writing the summary. New eye-tracking infrastructure allows for this improvement in the study environment. Data collected includes eye gazes on source code, written summaries, and time to complete each summary. Unlike prior work that concluded developers focus on the signature the most, these results indicate that they tend to focus on the method body more than the signature. Moreover, both experts and novices tend to revisit control flow terms rather than reading them for a long period. They also spend a significant amount of gaze time and have higher gaze visits when they read call terms. Experts tend to revisit the body of the method significantly more frequently than its signature as the size of the method increases. Moreover, experts tend to write their summaries from source code lines that they read the most.

Artefacts Package - First come first served: the impact of file position on code review

“…Although previous evidence [12] suggests that developers start a code review from the first file in the change-set, some factors might affect the order in which developers review files during code review. Given the findings of some eye-tracking studies (e.g., [2,19,46]), it seems plausible to think that some files in a code change might attract reviewers' attention more due to their content (e.g., files that contain more call terms, control flow terms) regardless of their position. For instance, Abid et al [2] found that developers visit call terms more often than non-call terms and spend the longest time reading call terms.…”

“…Given the findings of some eye-tracking studies (e.g., [2,19,46]), it seems plausible to think that some files in a code change might attract reviewers' attention more due to their content (e.g., files that contain more call terms, control flow terms) regardless of their position. For instance, Abid et al [2] found that developers visit call terms more often than non-call terms and spend the longest time reading call terms. The next most visited and read locations were control flow terms and signatures.…”

“…Code change. For the review task, we create a code change that satisfies the following requirements: (1) not belonging to any existing project (to avoid introducing bias caused by participants' previous knowledge of the review change-set); (2) written in Java, one of the most popular programming languages [56]; (3) self-contained (to minimize the previous knowledge participants need to understand the change fully); and (4) close to an actual code review scenario (to increase realism).…”

Revisiting the debate: Are code metrics useful for measuring maintenance effort?

“…User studies are subjective: Much metrics research relies on user studies to understand the impact of metrics on maintenance indicators (Abid et al 2019;Hofmeister et al 2017;Scalabrino et al 2016;Buse and Weimer 2010;Scalabrino et al 2017;Antinyan et al 2017;Bauer et al 2019;Kafura and Reddy 1987;Curtis et al 1979;Darcy et al 2005). One peril related to user studies is that the outcome often depends on human subjects, and can be inconclusive or even contradictory (Brittain 1982).…”

Towards Modeling Human Attention from Eye Movements for Neural Source Code Summarization

“…One way to measure the attention that humans pay to source code is by tracking the movement of a person's eyes when reading that code. A plethora of eye tracking studies has demonstrated consistent processes that programmers follow when reading code [1,2,13,47,53], and it has even been demonstrated that programmers who are blind seek the same information from code, just using a dierent mechanical process [5]. In general, people are very ecient at extracting key features they need to understand code.…”