Supporting and Accelerating Reproducible Research in Software Maintenance Using TraceLab Component Library

Dit, Bogdan; Moritz, Evan; Linares‐Vásquez, Mario; Poshyvanyk, Denys

doi:10.1109/icsm.2013.44

Cited by 12 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, there are specialised tools that support RR in a specific context such as the Component Library developed by Dit et al [10] to support software maintenance studies. However, in this paper, we concentrate on discussing the tools we ourselves have used.…”

Section: A Reproducible Research Environmentmentioning

confidence: 99%

See 1 more Smart Citation

Would wider adoption of reproducible research be beneficial for empirical software engineering research?

Madeyski

Kitchenham

2017

IFS

View full text Add to dashboard Cite

Abstract.Researchers have identified problems with the validity of software engineering research findings. In particular, it is often impossible to reproduce data analyses, due to lack of raw data, or sufficient summary statistics, or undefined analysis procedures. The aim of this paper is to raise awareness of the problems caused by unreproducible research in software engineering and to discuss the concept of reproducible research (RR) as a mechanism to address these problems. RR is the idea that the outcome of research is both a paper and its computational environment. We report some recent studies that have cast doubts on the reliability of research outcomes in software engineering. Then we discuss the use of RR as a means of addressing these problems. We discuss the use of RR in software engineering research and present the methodology we have used to adopt RR principles. We report a small working example of how to create reproducible research. We summarise advantages of and problems with adopting RR methods. We conclude that RR supports good scientific practice and would help to address some of the problems found in empirical software engineering research.

show abstract

Section: A Reproducible Research Environmentmentioning

confidence: 99%

“…In the context of software maintenance research, Dit et al [10] have constructed a publicly available library of components and experiments aiming to improve the reproducibility and extensibility of software maintenance experiments.…”

Section: Reproducibility In Software Engineeringmentioning

confidence: 99%

Would wider adoption of reproducible research be beneficial for empirical software engineering research?

Madeyski

Kitchenham

2017

IFS

View full text Add to dashboard Cite

show abstract

“…This chapter is an extension of our previous work [47] It is obvious that our effort does not cover the entire range of SM papers or techniques.…”

Section: Discussionmentioning

confidence: 99%

Configuring and Assembling Information Retrieval Based Solutions for Software Engineering Tasks

Dit

2016

2016 IEEE International Conference on Software Maintenance and Evolution (ICSME)

Self Cite

View full text Add to dashboard Cite

Information Retrieval (IR) approaches are used to leverage textual or unstructured data generated during the software development process to support various software engineering (SE) tasks (e.g., concept location, traceability link recovery, change impact analysis, etc.). Two of the most important steps for applying IR techniques to support SE tasks are preprocessing the corpus and configuring the IR technique, and these steps can significantly influence the outcome and the amount of effort developers have to spend for these maintenance tasks. We present the use of Genetic Algorithms (GAs) to automatically configure and assemble an IR process to support SE tasks. The approach named IR-GA determines the (near) optimal solution to be used for each step of the IR process without requiring any training. We applied IR-GA on three different SE tasks and the results of the study indicate that IR-GA outperforms approaches previously used in the literature, and that it does not significantly differ from an ideal upper bound that could be achieved by a supervised approach and a combinatorial approach

show abstract

“…Dit et al [14] surveyed software maintenance papers from the last ten years. Their goal was to reproduce the tools and results of these papers in the TraceLab framework, then make the components publicly available.…”

Section: Empirical Software Engineeringmentioning

confidence: 99%

Validation of Software Testing Experiments: A Meta-Analysis of ICST 2013

Hays

Hayes

Bathke

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Researchers in software testing are often faced with the following problem of empirical validation: does a new testing technique actually help analysts find more faults than some baseline method? Researchers evaluate their contribution using statistics to refute the null hypothesis that their technique is no better at finding faults than the state of the art. The decision as to which statistical methods are appropriate is best left to an expert statistician, but the reality is that software testing researchers often don't have this luxury. We developed an algorithm, MeansTest, to help automate some aspects of statistical analysis. We implemented MeansTest in the statistical software environment R, encouraging reuse and decreasing the need to write and test statistical analysis code. Our experiment showed that MeansTest has significantly higher F-measures than several other common hypothesis tests. We applied MeansTest to systematically validate the work presented at the 2013 IEEE Sixth International Conference on Software Testing, Verification, and Validation (ICST'13). We found six papers that potentially misstated the significance of their results. MeansTest provides a free and easy-to-use possibility for researchers to check whether their chosen statistical methods and the results obtained are plausible. It is available for download at coest.org.

show abstract

Supporting and Accelerating Reproducible Research in Software Maintenance Using TraceLab Component Library

Cited by 12 publications

References 42 publications

Would wider adoption of reproducible research be beneficial for empirical software engineering research?

Would wider adoption of reproducible research be beneficial for empirical software engineering research?

Configuring and Assembling Information Retrieval Based Solutions for Software Engineering Tasks

Validation of Software Testing Experiments: A Meta-Analysis of ICST 2013

Contact Info

Product

Resources

About