An empirical comparative evaluation of requirements engineering methods

Abstract

Requirements Engineering (RE) is a relatively young discipline, and still many advances have been achieved during the last decades. In particular, numerous RE approaches are proposed in the literature with the aim of understanding a certain problem (e.g. information systems development) and establishing a knowledge base that is shared between domain experts and developers (i.e. a requirements specification). However, there is a growing concern for empirical validations that assess RE proposals and statements. This paper is related to the assessment of the quality of functional requirements specifications, using the Method Evaluation Model (MEM) as a theoretical framework. The MEM distinguishes the actual efficacy and the perceived efficacy of a method. In order to assess the actual efficacy or RE methods, the conceptual model quality framework by Lindland et al. can be applied; in this paper, we focus on the completeness and granularity of requirements models and extend this framework by defining four new metrics (e.g. degree of functional encapsulations completeness with respect This paper revises and extends previous work that has been published in the 17th IEEE International Requirements Engineering Conference (RE'09) [1].S. España ( ) · N. Condori-Fernandez · Ó. Pastor to a reference model, number of functional fragmentation errors). In order to assess the perceived efficacy, conventional questionnaires can be used. A laboratory experiment with master students has been carried out, in order to compare (using the proposed metrics) two RE methods; namely, Use Cases and Communication Analysis. With respect to actual efficacy, results indicate greater model quality (in terms of completeness and granularity) when Communication Analysis guidelines are followed. With respect to perceived efficacy, we found that Use Cases was perceived to be slightly easier to use than Communication Analysis. However, Communication Analysis was perceived to be more useful in terms of determining the proper business processes granularity. The paper discusses these results and highlights some key issues for future research in this area.

Formalizing and Automating Use Case Model Development

“…Each factor was objectively measured against quality metrics. In this paper we make reference to what we and others [41,42] consider the most significant quality factors, namely completeness, correctness, and consistency. Table 5 shows the summarized results for the quality of each produced specification, followed by discussion only on completeness, due to space limitation.…”

“…Correctness. Correctness refers to the extent to which the model conforms to the rules and conventions of the writing/modeling technique [41], which are, in our case, the naming rules, definition rules, diagrammatic conventions, etc. for the creation of the use case elements, use-case diagrams and use-case specifications.…”

Control flow versus communication: comparing two approaches to process modelling

“…Lindland et al (1994) framework has been empirically examined and used for evaluating the comparative effectiveness of different modelling languages (e.g. España et al, 2010;Rockwell and Bajaj, 2010).…”

Methodological Integration of Communication Analysis Into a Model-Driven Software Development Framework

“…The material resulting from this lab demo has later been used as teaching material by Sergio España in the course "Model-driven Software Development" (2009) at University of Twente and in course "Software Technology for Web Environments" (2010 and 2011) at Universitat Politècnica de València. Also, it is part of the instrumentation of the experiments described in Section 6.5 (results published in [España, Condori-Fernández et al 2009] and [España, Condori-Fernández et al 2010]) and in Section 7.5.…”

“…International journal articles  España, S., N. Condori-Fernández, A. González and O. Pastor (2010). ʺAn Evaluating the completeness and granularity of functional requirements specifications: a controlled experiment.…”