An Empirical Analysis of Linear Adaptation Techniques for Case-Based Prediction

Abstract

In this paper we explore the role of case adaptation for feature vector prediction problems. We focus on software project effort. We study three data sets that range from small (less than 20 cases) through medium (approximately 80 cases) to large (approximately 400 cases). These are typical sizes for this problem domain. We compare two variants of a linear size adjustment technique and (as a baseline) a simple k-NN approach. Our results show that the linear scaling techniques studied result in statistically significant improvements to predictions. However, the size of these improvements is relatively small, typically about 10%. The results include a number of extreme outliers which might be problematic if the techniques are to be used in practice. This suggests further work is required to cope better with the outlier problem.Null adaptation, the simplest, involves directly applying the solution from the retrieved case(s) to the target case. This is the approach adopted by a simple

LSA-X: Exploiting Productivity Factors in Linear Size Adaptation for Analogy-Based Software Effort Estimation

“…In this calculation, the software size variable may be measured in terms such as adjusted Function Points, raw Function Points, or lines of code. Walkerden and Jeffery [5], based on their [5] Linear size adaptation Linear Software size 4 MSA [6] Multiple size adaptation Linear Software size(s) 5 RTM [7] Regression towards the mean Linear Software size 6 AQUA [8] Similarity-based adaptation Linear All features 7 GA [9] Adaptation based on Linear All features Genetic algorithm 8 NNet [10] Non-linear adaptation Non-linear All features based on Neural network observation, suggested that an effort adaptation using only a software size variable is rational, because any size variables are always strongly correlated with the effort. Furthermore, adjusting the effort using a size variable also allows an estimation to scale the estimated effort value between pairs of analogues based on differences in size.…”

“…Multiple Size Adaptation (MSA) extends the LSA technique to handle the case where the software size is elaborated with multiple arbitrary attributes, such as in web application development [6]. MSA aggregates multiple size variables using the mean and applies the mean software size to Eq.…”

“…Compared to the basic UAVG technique, the LSA technique generally produces a higher accuracy [5], [6], [16] because it can capture and exploit the degrees of difference between project cases with similar features, but different software sizes. LSA applies the degree of the difference in software size to the estimated effort value and scales the estimated effort to match the characteristics of the new case.…”

“…Thus, a different training/test split of a dataset may produce a different k value. This approach is considerably more intuitive than fixing a single k value for the entire dataset, which is commonly used in the software effort estimation literature [5], [6], [10]. In addition, selection of the k value in this way also allows us to omit the sensitivity analysis of the effectiveness of the adaptation techniques regarding the number of analogues, because the accuracy is no longer subject to an arbitrary assignment of a single k value.…”

“…Various adaptation techniques have been proposed in the literature such as the Linear Size Adaptation (LSA) technique [5], and many others [6]- [10]. However, despite being essential to analogy-based estimation, there is no generally accepted "best" solution adaptation technique.…”