An empirical analysis of data preprocessing for machine learning-based software cost estimation

Abstract

Context: Due to the complex nature of the software development process, traditional parametric models and statistical methods often appear to be inadequate to model the increasingly complicated relationship between project development cost and the project features (or cost drivers). Machine learning (ML) methods, with several reported successful applications, have gained popularity for software cost estimation in recent years. Data preprocessing has been claimed by many researchers as a fundamental stage of ML methods; however, very few works have been focused on the effects of data preprocessing techniques. Objective: This study aims for a systematic assessment of the effectiveness of data preprocessing techniques on ML methods in the context of software cost estimation. Method: In this work, we first conduct a literature survey of the recent publications using data preprocessing techniques, followed by a systematic empirical study to analyze the strengths and weaknesses of individual data preprocessing techniques as well as their combinations. Results: Our results indicate that data preprocessing techniques may significantly influence the final prediction. They sometimes might have negative impacts on prediction performance of ML methods. Conclusion: In order to reduce prediction errors and improve efficiency, a careful selection is necessary according to the characteristics of machine learning methods, as well as the datasets used for software cost estimation.

Cross-validation based K nearest neighbor imputation for software quality datasets: An empirical study

“…Less attention has been focused on MDT methods themselves. In a more recent study, Huang et al (2015) found that only some of the former software effort estimation studies have considered the significance of the MDTs, of which only Minku and Yao (2011) (Myrtveit et al, 2001;Strike et al, 2001), and the prediction error may be introduced (Mittas and Angelis, 2010). MEI is efficient and has been involved in SEE as the most popular imputation approach; however, it will cause bias to data.…”

“…For example, a well-known technique called listwise deletion, had been widely adopted for handling missing values during data-preprocessing, but it potentially impairs the completeness of data and introduces undesirable biases in estimation (Huang et al, 2015). By contrast, missing data imputation methods replace missing variables by artificial estimates (Song et al, 2008); at the same time maintain the data completeness.…”

“…Huang et al (2015) has found that MEI monopolizes the imputation approaches in recent software effort estimation studies. A review of the other MDTs in SEE studies is presented at last.…”

A genetic algorithm based framework for software effort prediction

“…They concluded that regression trees or analogy-based methods are the best performers and offered means to address the conclusion instability issue. In (Huang et al 2015) several data preprocessing techniques were empirically assessed on the effectiveness of machine learning methods for effort estimation. The results indicate that data preprocessing techniques may significantly influence the predictions, but sometimes it might have negative impacts on prediction performance.…”

“…Software project managers need to be able to estimate the effort and cost of development early in the life cycle, as it affects the success of software project management (Huang et al 2015).…”