An experiment for evaluating the effectiveness of using a system dynamics simulation model in software project management education

Abstract

Due to increasing demand for sofrware project managers in industry, efforts are needed to develop the management-related knowledge and skills of the current and future software workforce. In particular, university education needs to provide to their computer science students not only technology-related skills but, in addition, a basic understanding of typical phenomena occurring in industrial (and academic) sofrware projects.This paper presents a controlled experiment that evaluates the effectiveness of using a process simulation model for university education in sofrware project managemem. The experiment uses a pre-test-post-test control group design with random assignment of computer science students. The treatment of the experimental group involves a System Dynamics simulation model. The treatment of the control group involves a conventional predictive model for project planning, i.e. the well-known COCOMO model.In addition to the presentation of the results of the empirical study, the paper discusses limitations and threats to validity. Proposals for modifications of the experimental design and the treatments are made for future replications.

Effective Software Project Management Education through Simulation Models: An Externally Replicated Experiment

“…It is the degree to which the variables used in the study accurately measure the concepts they purport to measure. The related issues remain same as the KL experiment [11]. We highlight the points here.…”

“…However, more time was reserved during the replication as compared to the initial experiment. This was done in accordance with the recommendations of the original experiment [11]. Of the eleven students participating in the first replication, 6 were assigned randomly to the experimental group (A), and 5 to the control group (B).…”

“…A small number of empirical studies have been carried out to study the impact of software engineering education. One such study is by Pfahl et al [11] in which they have performed a controlled experiment to evaluate the learning effectiveness of using a process simulation model for educating computer science students in software project management. The experimental group applied a Systems Dynamics simulation model while the control group used the COCOMO model as a predictive tool for project planning.…”

“…University education must teach computer science and software-engineering students not only technology related skills but also a basic understanding of typical management phenomena occurring in industrial (and academic) software projects. Pfahl et al [11] have performed a controlled experiment that evaluates the effectiveness of using a process simulation model for university education in software project management. In their study, the experimental group applied a Systems Dynamics simulation model while the control group used the COCOMO model as a predictive tool for project planning.…”

“…In this paper, we discuss an external replication of the same experiment (of Pfahl et al [11]) which was performed at the University of Reading in England. Another externally replicated study has also been conducted at the University of Oulu, Finland [12].…”