The impact of continuous integration on other software development practices: A large-scale empirical study

Zhao, Yangyang; Serebrenik, Alexander; Zhou, Yuming; Filkov, Vladimir; Vasilescu, Bogdan

doi:10.1109/ase.2017.8115619

Cited by 117 publications

(120 citation statements)

References 41 publications

Supporting

Mentioning

116

Contrasting

Unclassified

Order By: Relevance

“…Results showed that most popular projects are using CI/CD and its adoption is continuously increasing. A similar approach showed that developers are improving automated tests after the adoption CI/CD (Zhao et al, 2017). Our work only focuses on the relation between automated tests and CI/CD in the context of mobile development, bringing some enlightenment on how the adoption of CI/CD differs in mobile app development.…”

Section: Related Workmentioning

confidence: 96%

See 1 more Smart Citation

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

View full text Add to dashboard Cite

Software testing is an important phase in the software development lifecycle because it helps in identifying bugs in a software system before it is shipped into the hand of its end users. There are numerous studies on how developers test general-purpose software applications. The idiosyncrasies of mobile software applications, however, set mobile apps apart from general-purpose systems (e.g., desktop, stand-alone applications, web services). This paper investigates working habits and challenges of mobile software developers with respect to testing. A key finding of our exhaustive study, using 1000 Android apps, demonstrates that mobile apps are still tested in a very ad hoc way, if tested at all. However, we show that, as in other types of software, testing increases the quality of apps (demonstrated in user ratings and number of code issues). Furthermore, we find evidence that tests are essential when it comes to engaging the community to contribute to mobile open source software. We discuss reasons and potential directions to address our findings. Yet another relevant finding of our study is that Continuous Integration and Continuous Deployment (CI/CD) pipelines are rare in the mobile apps world (only 26% of the apps are developed in projects employing CI/CD) -we argue that one of the main reasons is due to the lack of exhaustive and automatic testing.

show abstract

Section: Related Workmentioning

confidence: 96%

“…Why and How: Previous work showed the adoption of CI/CD with automated testing has beneficial results in software projects Hilton et al (2016); Zhao et al (2017). For that reason, the adoption of CI/CD is getting momentum in software projects.…”

Section: Rq4: How Does Automated Testing Affect Code Issues In Foss Amentioning

confidence: 99%

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…Zhao and colleagues [23] empirically investigated the adoption of Travis-CI in a large sample of GitHub projects. They quantitatively compared the CI transition in these projects using metrics such as commit frequency, code churn, pull request closing, and issue closing.…”

Section: Related Workmentioning

confidence: 99%

“…The main observations were: (i) a small increase in the number of merged commits after CI adoption; (ii) a statistically significant decreasing in the number of merge commit churn; (iii) a moderate increase in the number of issues closed after CI adoption; and (iv) a stationary behavior in the number of closed pull requests as well as a longer time to close PRs after the CI Adoption. Contrary to the work performed by Zhao and colleagues [23], we studied four scenarios of unhealthy CI usage instead of the impact that the adoption of CI may bring to a software project.…”

Section: Related Workmentioning

confidence: 99%

Continuous Integration Theater

Felidré

Furtado

Costa

et al. 2019

2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

View full text Add to dashboard Cite

Background: Continuous Integration (CI) systems are now the bedrock of several software development practices. Several tools such as TravisCI, CircleCI, and Hudson, that implement CI practices, are commonly adopted by software engineers. However, the way that software engineers use these tools could lead to what we call "Continuous Integration Theater", a situation in which software engineers do not employ these tools effectively, leading to unhealthy CI practices. Aims: The goal of this paper is to make sense of how commonplace are these unhealthy continuous integration practices being employed in practice. Method: By inspecting 1,270 open-source projects that use TravisCI, the most used CI service, we quantitatively studied how common is to use CI (1) with infrequent commits, (2) in a software project with poor test coverage, (3) with builds that stay broken for long periods, and (4) with builds that take too long to run. Results: We observed that 748 (∼60%) projects face infrequent commits, which essentially makes the merging process harder. Moreover, we were able to find code coverage information for 51 projects. The average code coverage was 78%, although Ruby projects have a higher code coverage than Java projects (86% and 63%, respectively). However, some projects with very small coverage (∼4%) were found. Still, we observed that 85% of the studied projects have at least one broken build that take more than four days to be fixed. Interestingly, very small projects (up to 1,000 lines of code) are the ones that take the longest to fix broken builds. Finally, we noted that, for the majority of the studied projects, the build is executed under the 10 minutes rule of thumb. Conclusions: Our results are important to an increasing community of software engineers that employ CI practices on daily basis but may not be aware of bad practices that are eventually employed.

show abstract

“…For instance, continuous integration and continuous development is a promising approach to address this issue. Although it is known to promote software best practices [32], they are rare in the mobile app world [33], [34]. In addition, results suggest that energy-related changes ought to be tackled by developers with additional care (e.g., code reviews).…”

Section: Discussionmentioning

confidence: 99%

Do Energy-Oriented Changes Hinder Maintainability?

Cruz

Abreu

Grundy

et al. 2019

2019 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Energy efficiency is a crucial quality requirement for mobile applications. However, improving energy efficiency is far from trivial as developers lack the knowledge and tools to aid in this activity. In this paper we study the impact of changes to improve energy efficiency on the maintainability of Android applications. Using a dataset containing 539 energy efficiencyoriented commits, we measure maintainability -as computed by the Software Improvement Group's web-based source code analysis service Better Code Hub (BCH) -before and after energy efficiency-related code changes. Results show that in general improving energy efficiency comes with a significant decrease in maintainability. This is particularly evident in code changes to accommodate the Power Save Mode and Wakelock Addition energy patterns. In addition, we perform manual analysis to assess how real examples of energy-oriented changes affect maintainability. Our results help mobile app developers to 1) avoid common maintainability issues when improving the energy efficiency of their apps; and 2) adopt development processes to build maintainable and energy-efficient code. We also support researchers by identifying challenges in mobile app development that still need to be addressed.

show abstract

The impact of continuous integration on other software development practices: A large-scale empirical study

Cited by 117 publications

References 41 publications

To the attention of mobile software developers: guess what, test your app!

To the attention of mobile software developers: guess what, test your app!

Continuous Integration Theater

Do Energy-Oriented Changes Hinder Maintainability?

Contact Info

Product

Resources

About