Missing data techniques in analogy-based software development effort estimation

Abnane

2017

Self Cite

Software development effort estimation (SDEE) plays a primary role in software project management. But choosing the appropriate SDEE technique remains elusive for many project managers and researchers. Moreover, the choice of a reliable estimation accuracy measure is crucial because SDEE techniques behave differently given different accuracy measures. The most widely used accuracy measures in SDEE are those based on magnitude of relative error (MRE) such as mean/median MRE (MMRE/MedMRE) and prediction at level p (Pred(p)), which counts the number of observations where an SDEE technique gave MREs lower than p. However, MRE has proven to be an unreliable accuracy measure, favoring SDEE techniques that underestimate. Consequently, an unbiased measure called standardized accuracy (SA) has been proposed. This paper deals with the Pred(p) and SA measures. We investigate (1) the consistency of Pred(p) and SA as accuracy measures and SDEE technique selectors, and (2) the relationship between Pred(p) and SA. The results suggest that Pred(p) is less biased towards underestimates and generally selects the same best technique as SA. Moreover, SA and Pred(p) measure different aspects of technique performance, and SA may be used as a predictor of Pred(p) by means of the 3 association rules.

Section: Empirical Designmentioning

confidence: 99%

Evaluating Pred(p) and standardized accuracy criteria in software development effort estimation

Abnane

2017

Self Cite

“…Tables B2‐B7 of Appendix present the descriptions of their selected attributes, respectively. For the ISBSG R8 dataset, we conducted a data pre‐processing phase to select data (projects and attributes) in order to only retain data with high quality . This phase consists of two steps: Project selection: …”

Section: Design Of Empirical Evaluationmentioning

confidence: 99%

“…The purpose of this step is to select historical projects with high quality data. Therefore, we used the following four criteria as in: Data quality rating: This field contains an ISBSG rating code of A, B, C, or D applied to the project data by the ISBSG quality reviewers. This code denotes the soundness and integrity of the data of each project.…”

Section: Design Of Empirical Evaluationmentioning

confidence: 99%

“…Thus, it can be easily adapted to deal with any attribute as a target by simply modifying which attributes will be considered in the distance measure. KNN can also handle examples with multiple missing values …”

Section: Introductionmentioning

confidence: 99%

“…To determine the KNN, the similarity between the incomplete/complete cases is measured by means of a given distance. The following distances have been used to find nearest cases: Euclidean distance: It measures the distance between two cases x and y described by n attributes using Equation .

bold-italicd ((),, x, y) bold= \sqrt{\sum_{j = 1}^{n} {(x_{j} - y_{j})}^{2}}

…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Support vector regression‐based imputation in analogy‐based software development effort estimation

Abnane

2018

Self Cite

Missing data (MD) is a widespread problem that can affect the ability to use data to construct effective software development effort estimation (SDEE) techniques. To deal with this challenge, several imputation techniques have been investigated in SDEE and k‐nearest neighbors (KNN)‐based imputation is still the most frequently used. To the best of our knowledge, no study has used support vector regression (SVR)‐based imputation to construct accurate estimation techniques, in particular those based on analogy. This paper introduces a new imputation technique based on SVR for handling MD in two analogy‐based SDEE techniques: classical analogy and fuzzy analogy. More specifically, we investigate whether the use of SVR instead of KNN in imputing MD improves the predictive performance of these two analogy‐based techniques. A total of 1134 experiments were conducted involving seven datasets, SVR/KNN MD imputation techniques (KNN with Euclidean and Manhattan distances), three missingness mechanisms (missing completely at random, missing at random, non‐ignorable missing), and MD percentages from 10% to 90%. The results suggest that the use of SVR imputation, rather than KNN imputation, may improve the prediction performance of both analogy‐based techniques. Furthermore, we found that the impact of MD percentage upon effort prediction performance is reduced when using SVR rather than KNN. Moreover, fuzzy analogy generates better estimates in terms of the standardized accuracy measure than classical analogy regardless of the MD technique, the dataset used, the missingness mechanism, or the MD percentage.

Evaluating filter fuzzy analogy homogenous ensembles for software development effort estimation

Hosni

2018

Self Cite

Researchers have developed and evaluated many techniques to deliver accurate estimates of the effort required to complete a new software program. Among these, analogy has emerged as a very promising technique, in particular the fuzzy analogy estimation technique that uses the fuzzy logic concepts in order to deal with both categorical and numerical data. The aim of this paper is twofold: (1) evaluate the impact of 3 filters on the predictive ability of single and ensemble fuzzy analogy techniques and (2) assess whether filters could be a source of diversity for fuzzy analogy ensembles. Moreover, it compares the filter single and ensemble fuzzy analogy techniques with fuzzy analogy ensembles built without using feature selection over 6 datasets. The overall results suggest that (1) more accurate estimates are generated when filters were used with single and ensemble fuzzy analogy techniques, (2) filter single fuzzy analogy techniques outperformed filter fuzzy analogy ensembles, and (3) fuzzy analogy ensembles without feature selection were more accurate than filter single and ensemble techniques. Therefore, though the use of feature selection techniques led single and ensemble fuzzy analogy to generate accurate estimations, they failed to be a source of diversity for fuzzy analogy ensembles. Hence, constructing fuzzy analogy homogenous ensembles that combine single fuzzy analogy techniques with different parameter configurations still generate better accuracy than filter fuzzy analogy ensembles. However, further empirical evaluations of filter/wrappers fuzzy analogy ensembles are required in order to confirm or refute these findings.