On the Relation of Test Smells to Software Code Quality

Spadini, Davide; Palomba, Fabio; Zaidman, Andy; Bruntink, Magiel; Bacchelli, Alberto

doi:10.1109/icsme.2018.00010

Cited by 112 publications

(122 citation statements)

References 76 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…Among all these studies, Khomh et al [52] showed that the presence of sub-optimal implementations in Java classes, i.e., code smells, has a strong impact on the likelihood that such classes will be often modified by developers. The results were later confirmed by several studies in the field [22,75,83,107], further highlighting the relevance of code smells for change-proneness. Our work is clearly based on these findings, and aims at providing additional evidence of how code smells can be adopted in the context of prediction models having the goal of identifying change-prone classes.…”

Section: Related Worksupporting

confidence: 64%

Improving change prediction models with code smell-related information

et al. 2019

Self Cite

View full text Add to dashboard Cite

Code smells represent sub-optimal implementation choices applied by developers when evolving software systems. The nagative impact of code smells has been widely investigated in the past: besides developers' productivity and ability to comprehend source code, researchers empirically showed that the presence of code smells heavily impacts the change-proneness of the affected classes. On the basis of these findings, in this paper we conjecture that code smell-related information can be effectively exploited to improve the performance of change prediction models, i.e., models having as goal that of indicating to developers which classes are more likely to change in the future, so that they may apply preventive maintenance actions. Specifically, we exploit the so-called intensity index -a previously defined metric that captures the severity of a code smell-and evaluate its contribution when added as additional feature in the context of three state of the art change prediction models based on product, process, and developer-based features. We also compare the performance achieved by the proposed model with the one of an alternative technique that considers the previously defined antipattern metrics, namely a set of indicators computed considering the history of code smells in files. Our results report that (i) the prediction performance of the intensity-including

show abstract

Section: Related Worksupporting

confidence: 64%

Improving change prediction models with code smell-related information

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our participants revealed that finding how to design a test code to avoid flakiness is an important challenge to face. This motivates the growing research area around test code quality [18,29,[33][34][35][36][37] and provides two promising directions that the research community can focus on: (i) the definition of a set of design patterns that can support the creation of deterministic tests; (ii) the definition of a set of flakiness-related anti-patterns that practitioners should avoid when writing test cases. While some initial steps have been done about the relation between test smells and flaky tests [31,32], further investigation is necessary.…”

Section: Discussionmentioning

confidence: 99%

Understanding flaky tests: the developer’s perspective

Eck

Palomba

Castelluccio³

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

Self Cite

117

101

View full text Add to dashboard Cite

Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite exercising unchanged code. In this work, we examine the perceptions of software developers about the nature, relevance, and challenges of flaky tests.We asked 21 professional developers to classify 200 flaky tests they previously fixed, in terms of the nature and the origin of the flakiness, as well as of the fixing effort. We also examined developers' fixing strategies. Subsequently, we conducted an online survey with 121 developers with a median industrial programming experience of five years. Our research shows that: The flakiness is due to several different causes, four of which have never been reported before, despite being the most costly to fix; flakiness is perceived as significant by the vast majority of developers, regardless of their team's size and project's domain, and it can have effects on resource allocation, scheduling, and the perceived reliability of the test suite; and the challenges developers report to face regard mostly the reproduction of the flaky behavior and the identification of the cause for the flakiness. Data and materials [https://doi.org/10.5281/zenodo.3265785]. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract

“…Test smells represent sub-optimal design or implementation choices applied by developers when defining test cases [15], [33], [34]. On the one hand, previous research showed that the presence of test smells can lead the test code to be less maintainable [35]- [37]. On the other hand, recent work demonstrated that test smells can be related to problems like test flakiness or fault-proneness of test and production code [37], [38].…”

Section: Test Smellsmentioning

confidence: 99%

“…On the one hand, previous research showed that the presence of test smells can lead the test code to be less maintainable [35]- [37]. On the other hand, recent work demonstrated that test smells can be related to problems like test flakiness or fault-proneness of test and production code [37], [38]. Thus, test smells may negatively influence the overall ability of a test case to find faults in production code.…”

Section: Test Smellsmentioning

confidence: 99%

“…Thus, test smells may negatively influence the overall ability of a test case to find faults in production code. We investigate 8 different test smell types originating from the catalog by van Deursen et al [15] and that, together with the others included in the catalog, have been shown to be either related to maintainability or effectiveness issues [35], [37], [38], i.e., Assertion Roulette, Eager Test, Lazy Test, Mystery Guest, Sensitive Equality, Resource Optimism, For Testers Only, and Indirect Testing. A description of these factors is available in the wiki page of our replication package [18].…”

Section: Test Smellsmentioning

confidence: 99%

See 1 more Smart Citation

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

Grano

Palomba

Gall

2021

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Test cases are crucial to help developers preventing the introduction of software faults. Unfortunately, not all the tests are properly designed or can effectively capture faults in production code. Some measures have been defined to assess test-case effectiveness: the most relevant one is the mutation score, which highlights the quality of a test by generating the so-called mutants, ie variations of the production code that make it faulty and that the test is supposed to identify. However, previous studies revealed that mutation analysis is extremely costly and hard to use in practice. The approaches proposed by researchers so far have not been able to provide practical gains in terms of mutation testing efficiency. This leaves the problem of efficiently assessing test-case effectiveness as still open. In this paper, we investigate a novel, orthogonal, and lightweight methodology to assess test-case effectiveness: in particular, we study the feasibility to exploit production and test-code-quality indicators to estimate the mutation score of a test case. We firstly select a set of 67 factors and study their relation with test-case effectiveness. Then, we devise a mutation score estimation model exploiting such factors and investigate its performance as well as its most relevant features. The key results of the study reveal that our estimation model only based on static features has 86% of both F-Measure and AUC-ROC. This means that we can estimate the test-case effectiveness, using source-code-quality indicators, with high accuracy and without executing the tests. As a consequence, we can provide a practical approach that is beyond the typical limitations of current mutation testing techniques.Abstract-Test cases are crucial to help developers preventing the introduction of software faults. Unfortunately, not all the tests are properly designed or can effectively capture faults in production code. Some measures have been defined to assess test-case effectiveness: the most relevant one is the mutation score, which highlights the quality of a test by generating the so-called mutants, i.e., variations of the production code that make it faulty and that the test is supposed to identify. However, previous studies revealed that mutation analysis is extremely costly and hard to use in practice. The approaches proposed by researchers so far have not been able to provide practical gains in terms of mutation testing efficiency. This leaves the problem of efficiently assessing test-case effectiveness as still open. In this paper, we investigate a novel, orthogonal, and lightweight methodology to assess test-case effectiveness: in particular, we study the feasibility to exploit production and test-code-quality indicators to estimate the mutation score of a test case. We firstly select a set of 67 factors and study their relation with test-case effectiveness. Then, we devise a mutation score estimation model exploiting such factors and investigate its performance as well as its most relevant features. The key results of the s...

show abstract

On the Relation of Test Smells to Software Code Quality

Cited by 112 publications

References 76 publications

Improving change prediction models with code smell-related information

Improving change prediction models with code smell-related information

Understanding flaky tests: the developer’s perspective

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

Contact Info

Product

Resources

About