A Time Series Analysis of TravisTorrent Builds: To Everything There Is a Season

Atchison, Abigail; Berardi, Christina; Best, Natalie; Stevens, Elizabeth; Linstead, Erik

doi:10.1109/msr.2017.29

Cited by 8 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The irregular component is computed as the residuals from the seasonal plus trend fit [34]. We use the STL algorithm as it does not assume any distribution of the time series, it has been successfully used in previous software engineering studies [30], [35], and an efficient implementation is available as an open-source R package 4 .…”

Section: Data Extractionmentioning

confidence: 99%

How Maintainability Issues of Android Apps Evolve

Malavolta

Verdecchia

Filipovic

et al. 2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Context. Android is the largest mobile platform today, with thousands of apps published and updated in the Google Play store everyday. Maintenance is an important factor in Android apps lifecycle, as it allows developers to constantly improve their apps and better tailor them to their user base. Goal. In this paper we investigate the evolution of various maintainability issues along the lifetime of Android apps. Method. We designed and conducted an empirical study on 434 GitHub repositories containing open, real (i.e., published in the Google Play store), and actively maintained Android apps. We statically analyzed 9,945 weekly snapshots of all apps for identifying their maintainability issues over time. We also identified maintainability hotspots along the lifetime of Android apps according to how their density of maintainability issues evolves over time. More than 2,000 GitHub commits belonging to identified hotspots have been manually categorized to understand the context in which maintainability hotspots occur. Results. Our results shed light on (i) how often various types of maintainability issues occur over the lifetime of Android apps, (ii) the evolution trends of the density of maintainability issues in Android apps, and (iii) an in-depth characterization of development activities related to maintainability hotspots. Together, these results can help Android developers in (i) better planning code refactoring sessions, (ii) better planning their code review sessions (e.g., steering the assignment of code reviews), and (iii) taking special care of their code quality when performing tasks belonging to activities highly correlated with maintainability issues. We also support researchers by objectively characterizing the state of the practice about maintainability of Android apps. Conclusions. Independently from the type of development activity, maintainability issues grow until they stabilize, but are never fully resolved.

show abstract

Section: Data Extractionmentioning

confidence: 99%

How Maintainability Issues of Android Apps Evolve

Malavolta

Verdecchia

Filipovic

et al. 2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

show abstract

“…However, these prediction techniques based on machine learning are affected by their own technical defects, such as feature extraction dependent on domain expert knowledge, model performance is affected by the correlation between selected features and data imbalance, and so on. In addition, the result data of CI build belong to time series data in essence 5 , but the current prediction techniques do not take time information into account, so the applicability of these prediction techniques based on machine learning is very limited.…”

Section: Introductionmentioning

confidence: 99%

Temporal convolutional network-based build result prediction for continuous integration

Cao¹

2023

International Conference on Cloud Computing, Performance Computing, and Deep Learning (CCPCDL 2023)

View full text Add to dashboard Cite

“…Furthermore, as logs of software development, the input data is naturally time series. Atchison et al [2] analyzed 1283 projects from TravisTorrent repository and found that CI/CD presents strong seasonal behavior. Consequently, it is reasonable to question the reliability of their prediction results.…”

Section: Introductionmentioning

confidence: 99%

“…The contribution of this study is three-fold: 1) We have conducted an experimental study to compare the time-series-validation and cross-validation under a class imbalance classification context. 2) We systematically compare the imbalanced and balanced learning methods using time-series-validation. 3) We provide a more comprehensive and realistic result for predictive CI/CD and reveal the reason of the phenomenon found in existing studies that the performance of prediction on different projects is quite different.…”

Section: Introductionmentioning

confidence: 99%

An Experimental Evaluation of Imbalanced Learning and Time-Series Validation in the Context of CI/CD Prediction

Liu

Zhang

Yang

et al. 2020

Proceedings of the Evaluation and Assessment in Software Engineering

View full text Add to dashboard Cite

Background: Machine Learning (ML) has been widely used as a powerful tool to support Software Engineering (SE). The fundamental assumptions of data characteristics required for specific ML methods have to be carefully considered prior to their applications in SE. Within the context of Continuous Integration (CI) and Continuous Deployment (CD) practices, there are two vital characteristics of data prone to be violated in SE research. First, the logs generated during CI/CD for training are imbalanced data, which is contrary to the principles of common balanced classifiers; second, these logs are also time-series data, which violates the assumption of cross-validation. Objective: We aim to systematically study the two data characteristics and further provide a comprehensive evaluation for predictive CI/CD with the data from real projects. Method: We conduct an experimental study that evaluates 67 CI/CD predictive models using both cross-validation and time-series-validation. Results: Our evaluation shows that crossvalidation makes the evaluation of the models optimistic in most cases, there are a few counterexamples as well. The performance of the top 10 imbalanced models are better than the balanced models in the predictions of failed builds, even for balanced data. The degree of data imbalance has a negative impact on prediction performance. Conclusion: In research and practice, the assumptions of the various ML methods should be seriously considered for the validity of research. Even if it is used to compare the relative performance of models, cross-validation may not be applicable to the problems with time-series features. The research community need to revisit the evaluation results reported in some existing research. CCS CONCEPTS • Software and its engineering → Software verification and validation; • Computing methodologies → Machine learning algorithms; Cross-validation.

show abstract

A Time Series Analysis of TravisTorrent Builds: To Everything There Is a Season

Cited by 8 publications

References 8 publications

How Maintainability Issues of Android Apps Evolve

How Maintainability Issues of Android Apps Evolve

Temporal convolutional network-based build result prediction for continuous integration

An Experimental Evaluation of Imbalanced Learning and Time-Series Validation in the Context of CI/CD Prediction

Contact Info

Product

Resources

About