Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining

Zaidman, Andy; Rompaey, Bart Van; Deursen, Arie van; Demeyer, Serge

doi:10.1007/s10664-010-9143-7

Cited by 130 publications

(118 citation statements)

References 54 publications

(49 reference statements)

Supporting

Mentioning

110

Contrasting

Unclassified

Order By: Relevance

“…As it can be seen, while the Code line smoothly increases over time, the Test line frequently stays constant across revisions, indicating that testing is often a phased activity [41], that takes place only at certain times during development. One exception is Git, where code and tests evolve more synchronously, with a large number of revisions modifying both code and tests.…”

Section: Code and Test Evolutionmentioning

confidence: 96%

“…Zaidman et al [41] have examined the co-evolution of code and tests on two open-source and one industrial Java applications. The study looks at the evolution of program coverage over time, but only computes coverage for the major and minor releases of each system, providing around ten data points for each system.…”

Section: Related Workmentioning

confidence: 99%

“…We suspect that the main reason behind the scarcity of studies focusing on dynamic metrics lies in the difficulty of running multiple software versions, 1 especially since doing so involves evolving dependencies and unstable (including non-compilable) versions. For example, prior work [41] cites the manual effort and the long time needed to run different revisions as the reason for reporting dynamic measurements for only a small number of versions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Marinescu

Hošek

Cadar

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Software repositories provide rich information about the construction and evolution of software systems. While static data that can be mined directly from version control systems has been extensively studied, dynamic metrics concerning the execution of the software have received much less attention, due to the inherent difficulty of running and monitoring a large number of software versions.In this paper, we present Covrig, a flexible infrastructure that can be used to run each version of a system in isolation and collect static and dynamic software metrics, using a lightweight virtual machine environment that can be deployed on a cluster of local or cloud machines.We use Covrig to conduct an empirical study examining how code and tests co-evolve in six popular open-source systems. We report the main characteristics of software patches, analyse the evolution of program and patch coverage, assess the impact of nondeterminism on the execution of test suites, and investigate whether the coverage of code containing bugs and bug fixes is higher than average.

show abstract

Section: Code and Test Evolutionmentioning

confidence: 96%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Marinescu

Hošek

Cadar

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…Even if the quality of test cases is important to ease their maintenance [1,23], it was shown that test smells, i.e., symptoms of possible design issues of the test code [22], are very spread both in open source and industrial code; moreover, they have a strong impact on comprehension and maintenance [3]. This situation is even worse in automatically generated test cases, since they are more affected by test smells than their manually written counterparts [16].…”

Section: Introductionmentioning

confidence: 99%

An empirical investigation on the readability of manual and generated test cases

Grano

Scalabrino

Gall

et al. 2018

Proceedings of the 26th Conference on Program Comprehension

View full text Add to dashboard Cite

Software testing is one of the most crucial tasks in the typical development process. Developers are usually required to write unit test cases for the code they implement. Since this is a time-consuming task, in last years many approaches and tools for automatic test case generation -such as EvoSuitehave been introduced. Nevertheless, developers have to maintain and evolve tests to sustain the changes in the source code; therefore, having readable test cases is important to ease such a process. However, it is still not clear whether developers make an effort in writing readable unit tests. Therefore, in this paper, we conduct an explorative study comparing the readability of manually written test cases with the classes they test. Moreover, we deepen such analysis looking at the readability of automatically generated test cases. Our results suggest that developers tend to neglect the readability of test cases and that automatically generated test cases are generally even less readable than manually written ones. ABSTRACTSoftware testing is one of the most crucial tasks in the typical development process. Developers are usually required to write unit test cases for the code they implement. Since this is a time-consuming task, in last years many approaches and tools for automatic test case generation -such as EvoSuite-have been introduced. Nevertheless, developers have to maintain and evolve tests to sustain the changes in the source code; therefore, having readable test cases is important to ease such a process. However, it is still not clear whether developers make an effort in writing readable unit tests. Therefore, in this paper, we conduct an explorative study comparing the readability of manually written test cases with the classes they test. Moreover, we deepen such analysis looking at the readability of automatically generated test cases. Our results suggest that developers tend to neglect the readability of test cases and that automatically generated test cases are generally even less readable than manually written ones. CCS CONCEPTS• Software and its engineering → Maintaining software; Search-based software engineering;

show abstract

“…While automated test suites are helpful from a (continuous) integration and regression testing perspective, they lead to a substantial amount of test code [16]. Like production code, test code needs to be maintained, understood, and adjusted upon changes to production code or requirements [8,10,13].…”

Section: Introductionmentioning

confidence: 99%

Measuring Test Case Similarity to Support Test Suite Understanding

Greiler

Deursen

Zaidman

2012

Objects, Models, Components, Patterns

Self Cite

View full text Add to dashboard Cite

Abstract. In order to support test suite understanding, we investigate whether we can automatically derive relations between test cases. In particular, we search for trace-based similarities between (high-level) endto-end tests on the one hand and fine grained unit tests on the other. Our approach uses the shared word count metric to determine similarity. We evaluate our approach in two case studies and show which relations between end-to-end and unit tests are found by our approach, and how this information can be used to support test suite understanding.

show abstract

Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining

Cited by 130 publications

References 54 publications

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

An empirical investigation on the readability of manual and generated test cases

Measuring Test Case Similarity to Support Test Suite Understanding

Contact Info

Product

Resources

About