Evaluating Maintainability Prejudices with a Large-Scale Study of Open-Source Projects

Abstract

Exaggeration or context changes can render maintainability experience into prejudice. For example, JavaScript is often seen as least elegant language and hence of lowest maintainability. Such prejudice should not guide decisions without prior empirical validation. We formulated 10 hypotheses about maintainability based on prejudices and test them in a large set of open-source projects (6,897 GitHub repositories, 402 million lines, 5 programming languages). We operationalize maintainability with five static analysis metrics. We found that JavaScript code is not worse than other code, Java code shows higher maintainability than C# code and C code has longer methods than other code. The quality of interface documentation is better in Java code than in other code. Code developed by teams is not of higher and large code bases not of lower maintainability. Projects with high maintainability are not more popular or more often forked. Overall, most hypotheses are not supported by open-source data.

Understanding the Impact of Development Efforts in Code Quality

“…On the one hand, the obtained results show that some patterns are aligned with results of some previous work where a significant difference in code quality regarding number of committers cannot be demonstrated [Norick, Krohn et al, 2010, Roehm, Veihelmann et al, 2019, Voulgaropoulou, Spanos et al, 2012. On the other hand, the correlation for other clusters was stronger as other work previously suggested [Perez-Castillo, Piattini et al, 2018].…”

“…[Voulgaropoulou, Spanos et al, 2012] draw similar conclusions after analysing various R statistical open-source systems. A more recent study by [Roehm, Veihelmann et al, 2019] rejected the hypothesis of code developed by a team has better quality than code developed by an individual, since the analysis thousands of GitHub repositories did not provide such evidence.…”

Analyzing Program Comprehensibility of Go Projects

“…The reason is that both of these languages are object-oriented languages [2] and both of these belong to the C-family (e.g., C++, C#) [8]. Furthermore, an existing study DOI reference number: 10.18293/SEKE2021-152 concluded that Java projects have higher program comprehensibility compared to C, C++ and C# projects [9]. This study measures program comprehensibility using five static code metrics namely Too Long Files, Too Long Methods, Nesting Depth, Lack of Cohesive Comments (non-informative comments) and Duplicate Comments [9], [10].…”

“…Furthermore, an existing study DOI reference number: 10.18293/SEKE2021-152 concluded that Java projects have higher program comprehensibility compared to C, C++ and C# projects [9]. This study measures program comprehensibility using five static code metrics namely Too Long Files, Too Long Methods, Nesting Depth, Lack of Cohesive Comments (non-informative comments) and Duplicate Comments [9], [10]. These metrics are language-independent and found to be a good indicator of program comprehensibility.…”

“…For analysis, 50 popular, diverse and open-source projects are selected from each language based on several criteria such as projects cannot be a fork, and it must have a description or readme file [9]. Results show that Go projects have significantly better comprehensibility than Java in terms of Nesting Depth, Lack of Cohesive Comments and Duplicate Comments.…”