Our goal in this study is to review the research related to the replication of empirical studies in software engineering in terms of replications of empirical studies and conceptual or theoretical work about replications. In this article we present the preliminary findings of this review, concentrating on the studies reporting replications and the related original studies. We applied the systematic review method to perform a mapping study about the current state of the replication work of empirical studies performed in software engineering research. We analyzed 16,126 articles, from which we extracted 93 articles reporting 125 replications performed between 1994 and 2010, of 76 original studies. Over 60% of the replications were performed in the last six years and 71% percent of the studies were internal replications. The topics of software construction, testing, and maintenance concentrate nearly 50% of the replication work, while software design, configuration management and software tools and methods are the topics with least replications. The number of replications grew in the last few years, but the absolute number of replications is still very small, in particular considering the breadth of topics in software engineering. Incentive to perform external replications and better standards to report empirical studies and their replications are still needed.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.