A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

Abstract

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

Findings from a multi-method study on test-driven development

“…In particular, the mental model created by the developers does not suit the granular nature of TDD-which, for example, is showed to have positive effects on software quality [68] . We propose the integration of another practice 7 to address this issue, alongside with a more granular specification of the requirements that would better fit the process.…”

An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development

“…Previous work [34] shows that TDD developers create a mental model of their solution to a task which is then translated into unit tests. Novice developers can be uncomfortable with such an activity due to the counter-intuitiveness of this step, but also due to the difficulty of writing tests of good granularity in the absence of the underlying production code [12,24]. Conversely, developers following the non-TDD approach can decide when and what to test without (mindlessly) following a process.…”

Developers’ Initial Perceptions on TDD Practice: A Thematic Analysis with Distinct Domains and Languages

“…This practice encourages developers to evolve the software through small steps. According to Fucci et al [12], an incremental test-last and TDD "could be substitutes and equally effective provided that they are performed at the same level of granularity and uniformity". Thus, they suggested that the quality assurance might not be tied to its test-first nature, but on granularity effect.…”