Evaluating the learning effectiveness of using simulations in software project management education: results from a twice replicated experiment

Abstract

The increasing demand for software project managers in industry requires strategies for the development of management-related knowledge and skills of the current and future software workforce. Although several educational approaches help to develop the necessary skills in a university setting, few empirical studies are currently available to characterise and compare their effects.This paper presents the results of a twice replicated experiment that evaluates the learning effectiveness of using a process simulation model for educating computer science students in software project management. While the experimental group applied a System Dynamics simulation model, the control group used the well-known COCOMO model as a predictive tool for project planning.The results of each empirical study indicate that students using the simulation model gain a better understanding about typical behaviour patterns of software development projects. The combination of the results from the initial experiment and the two replications with metaanalysis techniques corroborates this finding. Additional analysis shows that the observed effect can mainly be attributed to the use of the simulation model in combination with a web-based role-play scenario. This finding is strongly supported by information gathered from the debriefing questionnaires of subjects in the experimental group. They consistently rated the simulation-based role-play scenario as a very useful approach for learning about issues in software project management. q

Effective Software Project Management Education through Simulation Models: An Externally Replicated Experiment

“…Another externally replicated study has also been conducted at the University of Oulu, Finland [12]. We discuss here our findings and perform a meta-analysis over the results of the original experiment and the results of the two replications.…”

“…In this paper, we will discuss an externally replicated experiment in which we keep the design and the goal of the above experiment intact. We then analyse our results in relation to the original experiment and another externally replicated experiment, discussed in [12]. …”

Evaluation Methodologies

“…The internal validity is the degree to which conclusions can be drawn about the causal effect of the independent variable on the dependent variables. Potential threats include selection effects, non-random subject loss, instrumentation effect, and maturation effect (Pfahl, 2004).…”

Incorporation of Computational Fluid Dynamics into a Fluid Mechanics Curriculum

“…To investigate the effectiveness of introducing CFD laboratories into the fluid mechanics course, a controlled experiment applying a pre-test-post-test control group design was conducted (Pfahl et al, 2004). The students had to undertake two tests, one before the respective course (pre-test) and one after the respective course (post-test) with the introduction of CFD laboratories then being evaluated by comparing within-student posttest to pre-test scores, and by comparing the scores between students in the CFD group (A), i.e.…”