Test Code Quality and Its Relation to Issue Handling Performance

2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)

2017

Self Cite

Abstract-Regression testing is a core activity that allows developers to ensure that source code changes do not introduce bugs. An important prerequisite then is that test cases are deterministic. However, this is not always the case as some tests suffer from socalled flakiness. Flaky tests have serious consequences, as they can hide real bugs and increase software inspection costs. Existing research has focused on understanding the root causes of test flakiness and devising techniques to automatically fix flaky tests; a key area of investigation being concurrency. In this paper, we investigate the relationship between flaky tests and three previously defined test smells, namely Resource Optimism, Indirect Testing and Test Run War. We have set up a study involving 19,532 JUnit test methods belonging to 18 software systems. A key result of our investigation is that 54% of tests that are flaky contain a test code smell that can cause the flakiness. Moreover, we found that refactoring the test smells not only removed the design flaws, but also fixed all 54% of flaky tests causally co-occurring with test smells.

Section: Discussionmentioning

confidence: 68%

Notice of Retraction: Does Refactoring of Test Smells Induce Fixing Flaky Tests?

Palomba

2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)

2017

Self Cite

“…We also aim to improve the characterization of testing effort by making use of the recent test code quality model presented by Athanasiou et al [19].…”

Section: Discussionmentioning

confidence: 99%

Studying Fine-Grained Co-evolution Patterns of Production and Test Code

Marșavina

Romano

2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation

2014

Self Cite

Abstract-Numerous software development practices suggest updating the test code whenever the production code is changed. However, previous studies have shown that co-evolving test and production code is generally a difficult task that needs to be thoroughly investigated.In this paper we perform a study that, following a mixed methods approach, investigates fine-grained co-evolution patterns of production and test code. First, we mine fine-grained changes from the evolution of 5 open-source systems. Then, we use an association rule mining algorithm to generate the co-evolution patterns. Finally, we interpret the obtained patterns by performing a qualitative analysis.The results show 6 co-evolution patterns and provide insights into their appearance along the history of the analyzed software systems. Besides providing a better understanding of how test code evolves, these findings also help identify gaps in the test code thereby assisting both researchers and developers.

“…Poorly designed tests are known to have a negative impact on test maintenance, as they are more difficult to adjust when production code changes [7,30,47,33]. Automated tests first need to be maintained when they are generated, since testers need to manually validate each test case to check the assertions (oracle cost) [2,9].…”

Section: Introductionmentioning

confidence: 99%

Automatic test case generation: what if test code quality matters?

Palomba

Panichella

Proceedings of the 25th International Symposium on Software Testing and Analysis

et al. 2016

Self Cite

Test case generation tools that optimize code coverage have been extensively investigated. Recently, researchers have suggested to add other non-coverage criteria, such as memory consumption or readability, to increase the practical usefulness of generated tests. In this paper, we observe that test code quality metrics, and test cohesion and coupling in particular, are valuable candidates as additional criteria. Indeed, tests with low cohesion and/or high coupling have been shown to have a negative impact on future maintenance activities. In an exploratory investigation we show that most generated tests are indeed affected by poor test code quality. For this reason, we incorporate cohesion and coupling metrics into the main loop of search-based algorithm for test case generation. Through an empirical study we show that our approach is not only able to generate tests that are more cohesive and less coupled, but can (i) increase branch coverage up to 10% when enough time is given to the search and (ii) result in statistically shorter tests.