Evaluating filter fuzzy analogy homogenous ensembles for software development effort estimation

Abstract

Researchers have developed and evaluated many techniques to deliver accurate estimates of the effort required to complete a new software program. Among these, analogy has emerged as a very promising technique, in particular the fuzzy analogy estimation technique that uses the fuzzy logic concepts in order to deal with both categorical and numerical data. The aim of this paper is twofold: (1) evaluate the impact of 3 filters on the predictive ability of single and ensemble fuzzy analogy techniques and (2) assess whether filters could be a source of diversity for fuzzy analogy ensembles. Moreover, it compares the filter single and ensemble fuzzy analogy techniques with fuzzy analogy ensembles built without using feature selection over 6 datasets. The overall results suggest that (1) more accurate estimates are generated when filters were used with single and ensemble fuzzy analogy techniques, (2) filter single fuzzy analogy techniques outperformed filter fuzzy analogy ensembles, and (3) fuzzy analogy ensembles without feature selection were more accurate than filter single and ensemble techniques. Therefore, though the use of feature selection techniques led single and ensemble fuzzy analogy to generate accurate estimations, they failed to be a source of diversity for fuzzy analogy ensembles. Hence, constructing fuzzy analogy homogenous ensembles that combine single fuzzy analogy techniques with different parameter configurations still generate better accuracy than filter fuzzy analogy ensembles. However, further empirical evaluations of filter/wrappers fuzzy analogy ensembles are required in order to confirm or refute these findings.

Polynomial analogy‐based software development effort estimation using combined particle swarm optimization and simulated annealing

“…Since early 1980s, a large number of methods have been presented to estimate the SDE required to develop a new software. 16 Among them, ABE methods based on the sense that similar projects may have similar efforts, have proven to provide accurate estimations. 17 ABE techniques mimic the human decision-making in solving complex problems, which makes the problem to be easily understood.…”

“…In filter methods, the FS process is carried out with respect to unsupervised performance measures, for example, information, similarity, consistency, distance, and statistical test. 16 Although filter techniques are fast in term of CPU running time, they cannot guarantee achieving high accuracy, as they do not use any learning model to evaluate feature subsets. To achieve the best accuracy, wrapper methods should be applied, wherein the performance measures of supervised learning (classification or regression) are used, for example, accuracy, sensitivity, mean absolute error, and root mean square error.…”

On the value of filter feature selection techniques in homogeneous ensembles effort estimation

“…The principal aims of performing the feature selection are improving the estimations performance of estimators, reducing the cost needed to build predictors, and give better explication of the obtained result. 28 Three categories of feature selection techniques are defined in the literature 39 :…”

“…This paper investigates the filter techniques because it is known that the cost needed to conduct the feature selection step by means of the wrapper and embedded techniques is much greater than using the filter techniques. 28,39,40 Moreover, the empirical studies conducted in the literature report that different filter techniques result in different subsets of features [39][40][41] ; thus three feature selection techniques-based filter were examined.…”

“…The output of a univariate filter feature selection is the ranking of features according to their relevance; hence selecting the number of features that will feed the predictive method is the main issue with these feature selection techniques. The literature suggests that selecting log 2 (N) attributes where N is the number of features available in the dataset 18,39,50,51 : this study used this recommendation.…”