Estimating Story Points from Issue Reports

Porru, Simone; Murgia, Alessandro; Demeyer, Serge; Marchesi, Michele; Tonelli, Roberto

doi:10.1145/2972958.2972959

Cited by 38 publications

(41 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Five years later, Porru et al [9] proposed to classify user stories into SP classes. Their approach uses features extracted from 4,908 user story descriptions recorded in Jira issue reports, collected from eight open-source projects.…”

Section: Related Workmentioning

confidence: 99%

“…In the final step, they used a regressor to map the deep representation into the SP estimate. They evaluated Deep-SE on 23,313 issues from 16 open-source projects and showed that it outperforms both baseline estimators and Porru et al's approach [9] based on Mean Absolute Error.…”

Section: Related Workmentioning

confidence: 99%

“…One of the readily available information that intelligent task-level effort estimation techniques can use is the user story, which is stored in title and description of the issue in an issue-tracking systems. For almost a decade, researchers proposed techniques to predict the effort of a task in SP using features extracted from user stories [5], [8], [9], [10], [11]. In most cases, these studies urged that their model would be a decision support system for expert estimators in Agile teams.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we replicate three out of six research questions investigated in the original study. 2 These research questions compare Deep-SE to three naïve baseline techniques, and to another task-level effort estimation technique introduced by Porru et al [9] (we refer to it as "TF/IDF-SE" herein), and they also investigate the suitability of Deep-SE for cross-project estimation.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep Learning for Agile Effort Estimation Have We Solved the Problem Yet?

Tawosi,

Moussa,

Sarro

2022

Preprint

View full text Add to dashboard Cite

In the last decade, several studies have proposed the use of automated techniques to estimate the effort of agile software development. In this paper we perform a close replication and extension of a seminal work proposing the use of Deep Learning for agile effort estimation (namely Deep-SE), which has set the state-of-the-art since. Specifically, we replicate three of the original research questions aiming at investigating the effectiveness of Deep-SE for both within-project and cross-project effort estimation. We benchmark Deep-SE against three baseline techniques (i.e., Random, Mean and Median effort prediction) and a previously proposed method to estimate agile software project development effort (dubbed TF/IDF-SE), as done in the original study. To this end, we use both the data from the original study and a new larger dataset of 31,960 issues, which we mined from 29 open-source projects. Using more data allows us to strengthen our confidence in the results and further mitigate the threat to the external validity of the study. We also extend the original study by investigating two additional research questions. One evaluates the accuracy of Deep-SE when the training set is augmented with issues from all other projects available in the repository at the time of estimation, and the other examines whether an expensive pre-training step used by the original Deep-SE, has any beneficial effect on its accuracy and convergence speed. The results of our replication show that Deep-SE outperforms the Median baseline estimator and TF/IDF-SE in only very few cases with statistical significance (8/42 and 9/32 cases, respectively), thus confounding previous findings on the efficacy of Deep-SE. The two additional RQs revealed that neither augmenting the training set nor pre-training Deep-SE play a role in improving its accuracy and convergence speed. These results suggest that using semantic similarity is not enough to differentiate user stories with respect to their story points; thus, future work has yet to explore and find new techniques and features that obtain accurate agile software development estimates.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning for Agile Effort Estimation Have We Solved the Problem Yet?

Tawosi,

Moussa,

Sarro

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For instance, in Satapathy and Rath [32] Decision Tree, Stochastic Gradient Boosting and Random Forest were compared to assess effort estimation given a dataset composed by 21 projects. Porru et al [29] used Support Vector Machine, K-Nearest Neighbors and Decision Tree for the same purpose given data from eight open source projects. Several Neural Networks were used in Panda et al [27].…”

Section: Background and Related Workmentioning

confidence: 99%

An Effort Estimation Support Tool for Agile Software Development: An Empirical Evaluation

Dantas

Costa

Vinicius³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Accurate effort estimation is an important part of the software process. In Agile Software Development, the techniques for predicting effort are mostly based on expert judgment, but there are approaches based on Machine Learning. The theme continues to be challenging and a subject of further studies given the difficulty of finding accurate solutions to the problem. This paper proposes and evaluates a tool based on the decision tree method for effort estimation in agile projects. We evaluated our tool given its accuracy and ease of use collecting data from four projects. To evaluate the accuracy, we compared the values of Magnitude of Relative Error from the teams' estimations with the values provided by the tool. To evaluate the ease of use, we used the Technology Acceptance Mode. The initial results show that the tool can be reliably used with minimal training. In terms of accuracy, the tool achieved lower error compared to the estimates provided by the teams (mean: 19.05% vs 33.32%), and the evaluation means in TAM were higher than 4.0 in ten of the eleven variables analyzed on a Likert scale. From this work, we conclude that estimation by decision tree is a viable technique that, at the very least, can be used by project managers to complement current estimation techniques.

show abstract