A flexible method for software effort estimation by analogy

Abstract

Effort estimation by analogy uses information from former similar projects to predict the effort for a new project. Existing analogy-based methods are limited by their inability to handle non-quantitative data and missing values. The accuracy of predictions needs improvement as well. In this paper, we propose a new flexible method called AQUA that is able to overcome the limitations of former methods. AQUA combines ideas from two known analogy-based estimation techniques: case-based reasoning and collaborative filtering. The method is applicable to predict effort related to any object at the requirement, feature, or project levels. Which are the main contributions of AQUA when compared to other methods? First, AQUA supports non-quantitative data by defining similarity measures for different data types. Second, it is able to tolerate missing values. Third, the results from an explorative study in this paper shows that the prediction accuracy is sensitive to both the number N of analogies (similar objects) taken for adaptation and the threshold T for the degree of similarity, which is true especially for larger data sets. A fixed and small number of analogies, as assumed in existing analogy-based methods, may not produce the best accuracy of prediction. Fourth, a flexible mechanism based on learning of existing data is proposed for determining the appropriate values of N and T likely to offer the best accuracy of prediction. New criteria to measure the quality of prediction are proposed. AQUA was validated against two internal and one public domain data sets with non-quantitative attributes and missing values. The obtained results are encouraging. In addition, Empir Software Eng (2007) 12:65-106 a comparative analysis with existing analogy-based estimation methods was conducted using three publicly available data sets that were used by these methods. In two of the three cases, AQUA outperformed all other methods.

LSA-X: Exploiting Productivity Factors in Linear Size Adaptation for Analogy-Based Software Effort Estimation

“…Pca (Principal component analysis [8]) uses a transformation function to map the entire dataset into a new lowerdimensional space, where remaining values are uncorrelated but they still retain the same variance as in the original dataset.…”

“…In this calculation, the software size variable may be measured in terms such as adjusted Function Points, raw Function Points, or lines of code. Walkerden and Jeffery [5], based on their [5] Linear size adaptation Linear Software size 4 MSA [6] Multiple size adaptation Linear Software size(s) 5 RTM [7] Regression towards the mean Linear Software size 6 AQUA [8] Similarity-based adaptation Linear All features 7 GA [9] Adaptation based on Linear All features Genetic algorithm 8 NNet [10] Non-linear adaptation Non-linear All features based on Neural network observation, suggested that an effort adaptation using only a software size variable is rational, because any size variables are always strongly correlated with the effort. Furthermore, adjusting the effort using a size variable also allows an estimation to scale the estimated effort value between pairs of analogues based on differences in size.…”

“…To enable and ease the reproduction of the experiments, we adopted a procedure that was used in many recent prominent studies [4], [8], [13], [30]. In this study, the method used to sample the training/test instances is based on leaveone-out approach [31] (a.k.a.…”

“…Jack-knifing, where each software project case is the test instance of a model built from all the remaining cases). The leave-one-out is our method of choice, because (1) it is widely used [4], [8], [13], [30], (2) it does not depend on any randomization, so that an evaluation over the same dataset will have the exact same result in every single run, and (3) a recent study by Kocaguneli [32] compared leave-one-out with the widely used alternative N-way cross validation method, and suggested that the leave-one-out approach often generates lower estimation bias and is more robust for small and medium-sized datasets [13], [33] …”

“…The normalization for a feature vector i was produced by subtracting the minimum value of the vector from all the attribute values, then dividing result by value of the maximum minus the minimum values of i). This procedure is widely used in effort estimation studies such as in [4], [8], [37], [39], [40].…”