Potential and limitations of the ISBSG dataset in enhancing software engineering research: A mapping review

Abstract

Context: The International Software Benchmarking Standards Group (ISBSG) maintains a software development repository with over 6,000 software projects. This dataset makes it possible to estimate a project"s size, effort, duration, and cost. Objective: The aim of this study was to determine how and to what extent, ISBSG has been used by researchers from 2000, when the first papers were published, until June of 2012. Method: A systematic mapping review was used as the research method, which was applied to over 129 papers obtained after the filtering process. Results: The papers were published in 19 journals and 40 conferences. Thirty-five percent of the papers published between years 2000 and 2011 have received at least one citation in journals and only five papers have received six or more citations. Effort variable is the focus of 70.5% of the papers, 22.5% center their research in a variable different from effort and 7% do not consider any target variable. Additionally, in as many as 70.5% of papers, effort estimation is the research topic, followed by dataset properties (36.4%). The more frequent methods are Regression (61.2 %), Machine Learning (35.7%), and Estimation by Analogy (22.5%). ISBSG is used as the only support in 55% of the papers while the remaining papers use complementary datasets. The ISBSG release 10 is used most frequently with 32 references. Finally, some benefits and drawbacks of the usage of ISBSG have been highlighted. Conclusion: This work presents a snapshot of the existing usage of ISBSG in software development research. ISBSG offers a wealth of information regarding practices from a wide range of organizations, applications, and development types, which constitutes its main potential. However, a data preparation process is required before any analysis. Lastly, the potential of ISBSG to develop new research is also outlined. Keywords

Software Development Effort Estimation from Unstructured Software Project Description by Sequence Models

“…3.00 and background management authority settings To provide our clients a set of complete advanced function and good user experience vehicle-mounted recreational and multimedia system. 7.00 To develop a universal interface module based on Modbus communication agreement, makes each isomerous station can be connected.…”

Effort and Cost in Software Engineering

“…Lokan et al [32] position ISBSG as a 'low cost initial determination of an organization's industry position and its comparative strengths and weaknesses'. Fernández-Diego et al [33] inventoried the use of the ISBSG-repository by performing a systematic mapping review on 129 research papers. They found that in 70,5% of the studies prediction of effort is the main focus.…”

Predictive accuracy comparison between neural networks and statistical regression for development effort of software projects

“…The most common research topic when the ISBSG data set has been used is prediction methods with 70.5%. This percentage was obtained from 129 studies published between years 2000 and 2012 in 19 journals and 40 conferences [33].…”