What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

Abstract

Context Previous studies have characterized code comments in various programming languages, showing how high quality of code comments is crucial to support program comprehension activities, and to improve the effectiveness of maintenance tasks. However, very few studies have focused on understanding developer practices to write comments. None of them has compared such developer practices to the standard comment guidelines to study the extent to which developers follow the guidelines. Objective Therefore, our goal is to investigate developer commenting practices and compare them to the comment guidelines. Method This paper reports the first empirical study investigating commenting practices in Pharo Smalltalk. First, we analyze class comment evolution over seven Pharo versions. Then, we quantitatively and qualitatively investigate the information types embedded in class comments. Finally, we study the adherence of developer commenting practices to the official class comment template over Pharo versions. Results Our results show that there is a rapid increase in class comments in the initial three Pharo versions, while in subsequent versions developers added comments to both new and old classes, thus maintaining a similar code to comment ratio. We furthermore found three times as many information types in class comments as those suggested by the template. However, the information types suggested by the template tend to be present more often than other types of information. Additionally, we find that a substantial proportion of comments follow the writing style of the template in writing these information types, but they are written and formatted in a non-uniform way. Conclusion The results suggest the need to standardize the commenting guidelines for formatting the text, and to provide headers for the different information types to ensure a consistent style and to identify the information easily. Given the importance of high-quality code comments, we draw numerous implications for developers and researchers to improve the support for comment quality assessment tools.

Replication package for the paper "Do Comments follow Commenting Conventions? A case study in Java and Python"

“…Style guidelines suggest various comment conventions, but developers do not or rarely adopt them while writing comments. Of the rules that are applicable as shown in Figure 3, writing style and content rules were more often followed than syntax and structure rules, confirming previous results for Smalltalk [8]. It shows that developers are interested in writing informative and consistent comments.…”

“…When the conventions are applicable, developers often follow the writing style and content conventions (80% of comments), but violate structure conventions in Java and Python class comments (nearly 30% of comments), confirming the previous results for Smalltalk by Rani et al [8]. Although the projectspecific guidelines provide very few additional class comment conventions, these conventions are followed more often compared to the conventions suggested by the standard guidelines both in Java and Python class comments.…”

“…As high-quality comments support developers in understanding and maintaining their programs, it is essential to ensure the adherence of their comments to the style guidelines to evaluate the overall comment quality. Rani et al have investigated class comments of Smalltalk and their adherence to the commenting conventions provided by a default template [8]. They found that Smalltalk developers follow writing style and content-related comment conventions more than 50% of the time, but they use inconsistent structure and formatting of comment content.…”

“…STUDY DESIGN Data collection. Rani et al have previously investigated the adherence of Smalltalk class comments to the conventions provided by a default template [8]. To verify their results across other languages, we analyze class comments in Java and Python.…”

“…The rationale behind the taxonomy is that the approaches evaluating comment quality can focus on a specific aspect of comments they want to evaluate and improve. Categories such as Content and Writing Style, are considered important by Rani et al in their work on evaluating Smalltalk comments[8].According to Rani et al, the Content category contains the rules that describe which type of information the comment should contain while the Writing Style category contains natural-language specific rules, such as grammar, punctuations, and capitalization. We added three more categories, following their methodology, to cover other aspects of comments.…”