What Do We Know about Test-Driven Development?

Shull, Forrest; Melnik, Grigori; Turhan, Burak; Layman, Lucas; Diep, Madeline; Erdogmus, Hakan

doi:10.1109/ms.2010.152

Cited by 68 publications

(72 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The SE research community has taken an interest in investigating the effects of TDD on several outcomes, including testing effort, external software quality and developers' productivity. In particular, there is a plethora of controlled experiments (e.g., [10,18,27,28]), aggregated in secondary studies (e.g., [32,36,39]), which report mostly inconclusive, or inconsistent results. A number of experiments were carried out at the University of Oulu [13,16]-including replication studies [14,15]-investigating the same outcomes.…”

Section: Introductionmentioning

confidence: 99%

An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach

Fucci

Scanniello

Romano

et al. 2016

Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Self Cite

View full text Add to dashboard Cite

Context: Test-driven development (TDD) is an agile practice claimed to improve the quality of a software product, as well as the productivity of its developers. A previous study (i.e., baseline experiment) at the University of Oulu (Finland) compared TDD to a test-last development (TLD) approach through a randomized controlled trial. The results failed to support the claims. Goal: We want to validate the original study results by replicating it at the University of Basilicata (Italy), using a different design. Method: We replicated the baseline experiment, using a crossover design, with 21 graduate students. We kept the settings and context as close as possible to the baseline experiment. In order to limit researchers bias, we involved two other sites (UPM, Spain, and Brunel, UK) to conduct blind analysis of the data. Results: The Kruskal-Wallis tests did not show any significant difference between TDD and TLD in terms of testing effort (p-value = .27 ), external code quality (pvalue = .82 ), and developers' productivity (p-value = .83 ). Nevertheless, our data revealed a difference based on the order in which TDD and TLD were applied, though no carry over effect. Conclusions: We verify the baseline study results, yet our results raises concerns regarding the selection of experimental objects, particularly with respect to their interaction with the order in which of treatments are applied.We recommend future studies to survey the tasks used in experiments evaluating TDD. Finally, to lower the cost of replication studies and reduce researchers' bias, we encourage other research groups to adopt similar multi-site blind analysis approach described in this paper.

show abstract

Section: Introductionmentioning

confidence: 99%

An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach

Fucci

Scanniello

Romano

et al. 2016

Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…Debido al previsible reducido número de sujetos experimentales, los investigadores originales decidieron utilizar un diseño de medidas repetidas para aumentar el poder estadístico [8,9]. Dicho diseño se muestra en la Tabla 2.…”

Section: Diseñounclassified

Efectividad del Test-Driven Development: Un Experimento Replicado

Dieste

Fonseca²,

Raura³

et al. 2015

Revista Latinoamericana De Ingenieria De Software

View full text Add to dashboard Cite

Resumen-Los métodos ágiles y sus prácticas asociadas, e.g.: Test-Driven Developement (TDD), son ampliamente utilizadas en la industria y han sido repetidamente sometidas a estudios empíricos. Antecedentes: Se han realizado diversos experimentos en empresas y academia acerca de TDD. En general, los experimentos no muestran un efecto positivo de TDD en la calidad del código o la productividad de los programadores. Objetivo: Replicar el experimento UPM 2014 efectuado por N. Juristo y su equipo, para reproducir sus resultados y secundariamente, estudiar el efecto de la experiencia del desarrollador en la efectividad de TDD. I. INTRODUCCIÓNDesde su introducción en la década de los 90's, las metodologías ágiles han venido ganando adeptos y, hoy en día, se configuran como una de las aproximaciones más utilizadas para desarrollar software [1]. Las metodologías ágiles se basan en una serie de prácticas, tales como la programación por pares o el desarrollo dirigido por pruebas (TDD, por sus siglas en inglés) que prometen aumentar la calidad del producto software y la productividad de los programadores. Para comprobar dichas promesas, se han realizado múltiples estudios empíricos, los cuales ponen de manifiesto que estas prácticas pueden ser en ocasiones perjudiciales (por ejemplo: la programación por pares puede reducir la productividad) [2]. Asimismo, existe un buen número de aspectos que matizan o moderan la calidad y la productividad, tales como la complejidad del producto o la experiencia de los desarrolladores.TDD también ha sido objeto de múltiples estudios empíricos. Una reciente revisión sistemática [3] sugiere que TDD no posee efecto alguno, ni positivo ni negativo, sobre la productividad. Esto contrasta fuertemente con las opiniones de los evangelistas [4]. Adicionalmente, y a diferencia de la práctica de programación por pares, pocos estudios empíricos han estudiado posibles variables moderadoras (por ejemplo: la experiencia de los programadores, antes citada).Las limitaciones en el conocimiento científico acerca de TDD han propiciado que algunos investigadores lleven a cabo estudios experimentales en TDD. A este respecto, N. Juristo y su equipo de la Universidad Politécnica de Madrid (UPM) han iniciado una línea de investigación, en el marco de la cual vienen realizando experimentos en distintas universidades y empresas. Este artículo reporta la replicación de uno de los experimentos realizados en UPM en Marzo de 2014. Esta replicación se llevó a cabo en la Universidad de las Fuerzas Armadas ESPE Sede Latacunga (ESPEL) en Ecuador en Mayo 2014. De acuerdo a la tipología propuesta por Gómez et al.[5], esta replicación puede ser clasificada como literal, conjunta y externa. El propósito de llevar a cabo la replicación fue verificar los resultados del experimento original y secundariamente, estudiar el efecto de la experiencia del desarrollador en la efectividad de TDD.Los resultados de la replicación están en línea con los efectos observados en el experimento original. El uso de TDD no ha logrado aumentar la pro...

show abstract

“…As a result of this type of investigation, some studies point out that programmers experienced in TDD report that this practice increases the confidence in the result of their work and ensures a good design and fewer defects in code [7]. As a consequence, these factors collaborate to increase the quality of software [22].…”

Section: Introductionmentioning

confidence: 99%

Developers’ Initial Perceptions on TDD Practice: A Thematic Analysis with Distinct Domains and Languages

Choma

Guerra

Silva

2018

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

Abstract. Test-Driven Development (TDD) is one of the most popular agile practices among software developers. To investigate the software developers' initial perceptions when applying TDD, we have performed an exploratory study. This study was carried out with participants who had about ten years of professional experience (on average), the majority of whom with no experience using TDD. The study is in the context of an agile project course at the postgraduate level of a research institute. Participants individually developed medium size projects addressed to different domains and using different programming languages. Through a structured questionnaire with open and semi-open questions, we collected information on TDD effects such as the perceived benefits, encountered difficulties, and developer's opinion about the quality improvement of the software. Afterward, we conducted a thematic analysis of the qualitative data. Most participants noticed improvements in code quality, but few have a more comprehensive view of the effects of TDD on software design. Our findings suggest that after overcoming the initial difficulties to understand where to start, and know how to create a test for a feature that does not yet exist, participants gain greater confidence to implement new features and make changes due to broad test coverage.

show abstract

What Do We Know about Test-Driven Development?

Cited by 68 publications

References 1 publication

An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach

An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach

Efectividad del Test-Driven Development: Un Experimento Replicado

Developers’ Initial Perceptions on TDD Practice: A Thematic Analysis with Distinct Domains and Languages

Contact Info

Product

Resources

About