A Large-Scale Empirical Study on Code-Comment Inconsistencies

Wen, Fengcai; Nagy, Csaba; Bavota, Gabriele; Lanza, Michele

doi:10.1109/icpc.2019.00019

Cited by 71 publications

(36 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the relevance of code comments for program comprehension and maintenance activities (Woodfield et al 1981;Tenny 1985;Tenny 1988;Hartzman and Austin 1993;de Souza et al 2006;Lidwell et al 2010;Cornelissen et al 2009), researchers have analyzed comments to detect low-quality comments (Steidl et al 2013;Liu et al 2015), identify existing inconsistency between comments and their related code elements (Ratol and Robillard 2017;Wen et al 2019;Stylos et al 2009;Petrosyan et al 2015;Zhou et al 2017), and they have examined the co-evolution of comments and code (Jiang and Hassan 2006;Fluri et al 2007;Fluri et al 2009;Ibrahim et al , 2012). However, very few studies have focused on analyzing the information embedded in the source code comments (Padioleau et al 2009;Haouari et al 2011;Steidl et al 2013;Pascarella and Bacchelli 2017;Zhang et al 2018), and none of them specifically analyzed class comments, or to what extent these class commenting practices adhere to the coding style guidelines.…”

Section: Introductionmentioning

confidence: 99%

What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

et al. 2021

View full text Add to dashboard Cite

Context Previous studies have characterized code comments in various programming languages, showing how high quality of code comments is crucial to support program comprehension activities, and to improve the effectiveness of maintenance tasks. However, very few studies have focused on understanding developer practices to write comments. None of them has compared such developer practices to the standard comment guidelines to study the extent to which developers follow the guidelines. Objective Therefore, our goal is to investigate developer commenting practices and compare them to the comment guidelines. Method This paper reports the first empirical study investigating commenting practices in Pharo Smalltalk. First, we analyze class comment evolution over seven Pharo versions. Then, we quantitatively and qualitatively investigate the information types embedded in class comments. Finally, we study the adherence of developer commenting practices to the official class comment template over Pharo versions. Results Our results show that there is a rapid increase in class comments in the initial three Pharo versions, while in subsequent versions developers added comments to both new and old classes, thus maintaining a similar code to comment ratio. We furthermore found three times as many information types in class comments as those suggested by the template. However, the information types suggested by the template tend to be present more often than other types of information. Additionally, we find that a substantial proportion of comments follow the writing style of the template in writing these information types, but they are written and formatted in a non-uniform way. Conclusion The results suggest the need to standardize the commenting guidelines for formatting the text, and to provide headers for the different information types to ensure a consistent style and to identify the information easily. Given the importance of high-quality code comments, we draw numerous implications for developers and researchers to improve the support for comment quality assessment tools.

show abstract

Section: Introductionmentioning

confidence: 99%

What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Wen et al [18] presented a large-scale empirical study of codecomment inconsistencies, revealing causes such as deprecation and refactoring. Zhou et al [21,22] contributed a line of work on detecting defects of API documents with techniques from program comprehension and natural language processing.…”

Section: Background and Related Workmentioning

confidence: 99%

Beyond accuracy: assessing software documentation quality

Middleton

Atapattu

2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Good software documentation encourages good software engineering, but the meaning of "good" documentation is vaguely defined in the software engineering literature. To clarify this ambiguity, we draw on work from the data and information quality community to propose a framework that decomposes documentation quality into ten dimensions of structure, content, and style. To demonstrate its application, we recruited technical editors to apply the framework when evaluating examples from several genres of software documentation. We summarise their assessments-for example, reference documentation and README files excel in quality whereas blog articles have more problems-and we describe our vision for reasoning about software documentation quality and for the expansion and potential of a unified quality framework. CCS CONCEPTS • Software and its engineering → Documentation; • Social and professional topics → Quality assurance.

show abstract

“…Documentation Quality: While software documentation greatly supports development activities, research has also found quality issues in documentation. Several studies combined program analysis and natural language processing techniques to detect inconsistencies between source code and documentation [40,45,48,49], with the inconsistencies typically revealing incorrect or outdated information. Unlike these studies, our work compares performance concerns extracted from ocial documentation and crowd documentation.…”

Section: Knowledge In Documentationmentioning

confidence: 99%

Understanding performance concerns in the API documentation of data science libraries

Tao

Jiang

Liu

et al. 2020

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

The development of ecient data science applications is often impeded by unbearably long execution time and rapid RAM exhaustion. Since API documentation is the primary information source for troubleshooting, we investigate how performance concerns are documented in popular data science libraries. Our quantitative results reveal the prevalence of data science APIs that are documented in performance-related context and the infrequent maintenance activities on such documentation. Our qualitative analyses further reveal that crowd documentation like Stack Overow and GitHub are highly complementary to ocial documentation in terms of the API coverage, the knowledge distribution, as well as the specic information conveyed through performance-related content. Data science practitioners could benet from our ndings by learning a more targeted search strategy for resolving performance issues. Researchers can be more assured of the advantages of integrating both the ocial and the crowd documentation to achieve a holistic view on the performance concerns in data science development.

show abstract

A Large-Scale Empirical Study on Code-Comment Inconsistencies

Cited by 71 publications

References 29 publications

What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

Beyond accuracy: assessing software documentation quality

Understanding performance concerns in the API documentation of data science libraries

Contact Info

Product

Resources

About