Effort Estimation Based on Collaborative Filtering

Abstract

Effort estimation methods are one of the important tools for project managers in controlling human resources of ongoing or future software projects. The estimations require historical project data including process and product metrics that characterize past projects. Practically, in using the estimation methods, it is a problem that the historical project data frequently contain substantial missing values. In this paper, we propose an effort estimation method based on Collaborative Filtering for solving the problem. Collaborative Filtering has been developed in information retrieval researchers, as one of the estimation techniques using defective data, i.e. data having substantial missing values. The proposed method first evaluates similarity between a target (ongoing) project and each past project, using vector based similarity computation equation. Then it predicts the effort of the target project with the weighted sum of the efforts of past similar projects. We conducted an experimental case study to evaluate the estimation performance of the proposed method. The proposed method showed better performance than the conventional regression method when the data had substantial missing values.

The state‐of‐the‐art in software development effort estimation

“…The estimation accuracy of SDEE models depends on data sets and their properties. 3,24,29,30,35,40,43 During our study, we found that some of the data sets collected from industry are being frequently used by researchers; of course, the availability is the prominent reason of it. Some of the researchers have also used pilot studies or even student projects to build and assess their models such as in previous studies.…”

“…Data sets are the key ingredients to any SDEE model; after all, an SDEE model is nothing but the finest generalized approximation of the past experiences of previously completed projects so that it can be used for the estimation of new software. The estimation accuracy of SDEE models depends on data sets and their properties . During our study, we found that some of the data sets collected from industry are being frequently used by researchers; of course, the availability is the prominent reason of it.…”

“…There are studies that have performed assessment of their model's accuracy using a list of measures other than the conventional MRE‐based measures …”

“…120 There are studies that have performed assessment of their model's accuracy using a list of measures other than the conventional MRE-based measures. 30,48 To improve measurement of the effectiveness of SDEE models with an idea of comparing the estimates with random guessing, Shepperd et al 118 have come up with a very new accuracy measure, named "standardized accuracy" (SA). Unlike MMRE measure, SA is unbiased, and it uses mean absolute residual (MAR) as an indicator.…”

Proposal of a Complete Life Cycle In-Process Measurement Model Based on Evaluation of an In-Process Measurement Experiment Using a Standardized Requirement Definition Process

“…Using collaborative filtering technology with in-process measurements of the target project and a database of over 1000 past projects benchmark data, project estimates and predictions were made. Collaborative filtering is one of the estimation techniques using defective data having substantial missing values, in information retrieval research domain [4]. 3) Using the automatic measurement platform named "Empirical Project Monitor (EPM)," the program modification history from the configuration management system, faults and maintenance data from the bug tracking system, and project e-mail information from the mailing list management system were analyzed.…”