Continual Prediction of Bug-Fix Time Using Deep Learning-Based Activity Stream Embedding

Abstract

Predicting the fix time of a bug is important for managing the resources and release milestones of a software development project. However, it is considered non-trivial to achieve high accuracy when predicting bug-fix times. We view that such difficulties come from the lack of continuous or posterior estimation based on subsequent developers' activities after a bug is initially reported. In this paper, we formulate the problem of bug-fix time prediction into a continual update of estimates with more activities. Logging data of bug-related activities that are streamed to a bug tracking system change the bug reports, enabling us to recalculate predictions over time. To do so, we propose a deep learning-based two-staged activity stream embedding model, DASENet that employs (i) a merged network for extracting contextual features across different types of logs, and (ii) a sequence network for exploring temporal relations of the logs. Through experiments with bug tracking system datasets from open source projects including Firefox, Chromium, and Eclipse, we show that DASENet achieves stable performance, e.g., for the Firefox dataset, top-1 accuracy of 4.6 to 8.5 % higher than other state-of-the-art works. Our approach also provides a transferable structure, yielding robust performance with a small dataset for different tasks; the DASENet model trained with a small dataset of about 900 samples (2 % of a full dataset) can show competitive performance to the other models with a full dataset. To the best of our knowledge, we are the first to employ deep learning on log streams in the context of bug-fix time prediction. INDEX TERMS Bug-fix time, activity stream, bug tracking system, deep learning, activity embedding, sequence learning model.

A Versatile Dataset of Agile Open Source Software Projects

“…This period can be considered as an approximation of the time taken by the development team to resolve the issue. This is usually the target variable used for bug resolution/fixing time estimation [14,18,27]. Other proxies for time are provided, such as In_Progress_Time and Total_Effort_Time, indicating, respectively, the implementation time and the development (including code review and testing) time.…”

“…Additionally, the dataset provides other useful information that can be considered for optimising task assignment, for example, considering developers' work load [5]. The dataset also provides the issue status transitions, which can be used to analyse activities and events to predict the time to fix a bug, or bug triage [14,18,27].…”

FRUGAL: Unlocking SSL for Software Analytics

“…• Traditional or non-neural approaches include (1) Guo et al [27]'s study on a large closed-source project (Microsoft Windows) to predict whether or not a bug will be fixed; and (2) Marks et al [51] used ensemble method of decision trees, i.e., random forests, on Eclipse and Mozilla data. • As to deep learning or neural network approach are DASENet [42] and DeepTriage [50]. • Only a minority of deep learning papers (39.4%) performed any sort of hyper-parameter optimization, i.e., varied few numbers of parameters, such as the number of layers of the deep learner, to edge out the best performance of deep learning.…”

“…• Only a minority of deep learning papers (39.4%) performed any sort of hyper-parameter optimization, i.e., varied few numbers of parameters, such as the number of layers of the deep learner, to edge out the best performance of deep learning. Even fewer To obtain a fair comparison with the prior state-of-the-art, we use the same data as used in the Lee et al [42], Mani et al [50], Yedida et al [93]'s studies. The data was collected from the three projects of Firefox, Chromium, and Eclipse:…”

“…An overview of the data used in the Lee et al[42], Mani et al[50], and Yedida et al[93] studies. Note that because of the manner of data collection, i.e., using binsequences for each day for each report, there are many more data samples generated from the number of reports mined.…”