In search of evidence for model-driven development claims: An experiment on quality, effort, productivity and satisfaction

Abstract

Abstract. Context: Model-Driven Development (MDD) is a paradigm that prescribes building conceptual models that abstractly represent the system and generating code from these models through transformation rules. The literature is rife with claims about the benefits of MDD, but they are hardly supported by evidences. Objective: This experimental investigation aims to verify some of the most cited benefits of MDD. Method: We run an experiment on a small set of classes using student subjects to compare the quality, effort, productivity and satisfaction of traditional development and MDD. The experiment participants built two web applications from scratch, one where the developers implement the code by hand and another using an industrial MDD tool that automatically generates the code from a conceptual model. Results: Outcomes show that there are no significant differences between both methods with regard to effort, productivity and satisfaction, although quality in MDD is more robust to small variations in problem complexity. We discuss possible explanations for these results. Conclusions: For small systems and less programming-experienced subjects, MDD does not always yield better results than a traditional method, even regarding effort and productivity. This contradicts some previous statements about MDD advantages. The benefits of developing a system with MDD appear to depend on certain characteristics of the development context.

TraceME: A Traceability-Based Method for Conceptual Model Evolution

“…The benefits of MDD are very well known (Whittle, Hutchinson et al 2014), (Hutchinson, Rouncefield et al 2011), (Panach, España et al 2015):…”

Assessing the impact of Volatile Functionality removal in web applications: Model‐Driven vs Code‐Based approaches

“…The empirical study result highlighted that developers find Model‐Driven Development (MDD) more useful than Code‐Based approaches even through its learning curve, and the need to replicate the evaluation in other commercial and academic context. Another study regarding MDD performance against Code‐Based was performed by Panach et al, and it focused on verifying some of the most cited benefits of MDD. By means of empirical evaluations with students, they analyzed quality, effort, productivity, and satisfaction aspects on MDD and Code‐Based projects.…”

“…We use these metric as Groovy application is based on an Object‐Oriented Paradigm. Different empirical research works compare Code‐Based and Model‐Driven approaches for either developing applications from scratch or maintaining applications. However, they focus on developers' perception of the experience.…”

Considerations about quality in model-driven engineering

“…However, for the industrial context, specialized websites (grey literature 12 ) make significant contributions to the quality from a practitioners' perspective. We found 49 industrial MDE Quality evidence in industry MDE Quality evidence in academic & research Industrial issue 01 (Vara and Marcos, 2012;Cuadrado et al, 2014;Hutchinson et al, 2014;Bertrand Portier, 2009;Haan, 2008;Whittle et al, 2014;Dubray, 2011;Linders, 2015;Tone, 2010;Finnie, 2015;Klinke, 2008;Vallecillo, 2014;Hebig and Bendraou, 2014;Brown, 2009;DenHaan, 2009DenHaan, , 2011bPierson, 2007;Cabot, 2009;Brambilla, 2016;OMG, 2016;Mohagheghi et al, 2009b;Igarza et al, 2012;Clark and Muller, 2012;Whittle et al, 2015;Aranda et al, 2012;Baker et al, 2005;Whittle et al, 2013;Cuadrado et al, 2014;Hoang, 2012;Platania, 2016;Fournier, 2008) (Agner et al, 2013;Panach et al, 2015a;Davies et al, 2006;Mussbacher et al, 2014;Quintero and Muñoz, 2011;Quintero et al, 2012;Poruban et al, 2014;Bruel et al, 2015;…”