Tawosi, Vali;
Moussa, Rebecca;
Sarro, Federica;
(2022)
Agile Effort Estimation: Have We Solved the Problem Yet? Insights From A Replication Study.
IEEE Transactions on Software Engineering
10.1109/tse.2022.3228739.
(In press).
Preview |
Text
Tawosi_Agile_Effort_Estimation_Have_We_Solved_the_Problem_Yet_Insights_From_A_Replication_Study.pdf Download (457kB) | Preview |
Abstract
In the last decade, several studies have explored automated techniques to estimate the effort of agile software development. We perform a close replication and extension of a seminal work proposing the use of Deep Learning for Agile Effort Estimation (namely Deep-SE), which has set the state-of-the-art since. Specifically, we replicate three of the original research questions aiming at investigating the effectiveness of Deep-SE for both within-project and cross-project effort estimation. We benchmark Deep-SE against three baselines (i.e., Random, Mean and Median effort estimators) and a previously proposed method to estimate agile software project development effort (dubbed TF/IDF-SVM), as done in the original study. To this end, we use the data from the original study and an additional dataset of 31,960 issues mined from TAWOS, as using more data allows us to strengthen the confidence in the results, and to further mitigate external validity threats. The results of our replication show that Deep-SE outperforms the Median baseline estimator and TF/IDF-SVM in only very few cases with statistical significance (8/42 and 9/32 cases, respectively), thus confounding previous findings on the efficacy of Deep-SE. The two additional RQs revealed that neither augmenting the training set nor pre-training Deep-SE play lead to an improvement of its accuracy and convergence speed. These results suggest that using semantic similarity is not enough to differentiate user stories with respect to their story points; thus, future work has yet to explore and find new techniques and features that obtain accurate agile software development estimates.
| Type: | Article |
|---|---|
| Title: | Agile Effort Estimation: Have We Solved the Problem Yet? Insights From A Replication Study |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tse.2022.3228739 |
| Publisher version: | https://doi.org/10.1109/tse.2022.3228739 |
| Language: | English |
| Additional information: | This is an Open Access article published under a Creative Commons Attribution 4.0 International (CC BY 4.0) Licence (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Software Effort Estimation, Story Point Estimation, Deep Learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10161949 |
Archive Staff Only
 |
View Item |
