A framework for classifying and comparing software architecture evaluation methods

Abstract

Software architecture evaluation has been proposed as a means to achieve quality attributes such as maintainability and reliability in a system. The objective of the evaluation is to assess whether or not the architecture will lead to the desired quality attributes. Recently, there have been a number of evaluation methods proposed. There is, however, little consensus on the technical and non-technical issues that a method should comprehensively address and which of the existing methods is most suitable for a particular issue. This paper presents a set of commonly known but informally described features of an evaluation method and organizes them within a framework that should offer guidance on the choice of the most appropriate method for an evaluation exercise. In this paper, we use this framework to characterise eight SA evaluation methods.

Towards Independent Software Architecture Review

“…Early detection of problems in architectural design through evaluation techniques reduces development costs and improves the quality of systems [5,8,14]. Thus, improving the effectiveness of evaluation techniques (in terms of problem detection) for architectural design is important.…”

Validating a model-driven software architecture evaluation and improvement method: A family of experiments

“…These questions 4 The control questions are included in the booklet (which is available at http://users.dsic.upv.es/~jagonzalez/IST/family.html) and should be answered by the participants after helped the participants to focus on understanding the patterns and metrics and allowed us to control their comprehension of the problem. These questions did not influence the experiment execution and results; their purpose was solely to control the comprehension of the patterns and metrics.…”

“…Control questions: We included a set of control questions 4 in the experimental material in order to analyze the comprehension of the patterns and the metrics being applied. These questions 4 The control questions are included in the booklet (which is available at http://users.dsic.upv.es/~jagonzalez/IST/family.html) and should be answered by the participants after helped the participants to focus on understanding the patterns and metrics and allowed us to control their comprehension of the problem.…”

Mining patterns to support software architecture evaluation

“…Scenario-based SA evaluation methods such as ATAM, SAAM, and ALMA, are considered relatively mature and established as they have been widely applied and more rigorously validated in various domains [7]. Figure 1 shows a generic process of scenario-based SA evaluation.…”

“…A number of methods, such as Architecture Tradeoff Analysis Method (ATAM) [4], Software Architecture Analysis Method (SAAM) [5] and Architecture-Level Maintainability Analysis (ALMA) [6], have been developed to evaluate the quality related issues at the SA level. Most of these methods are structurally same but there are a number of differences among their activities and techniques [7]. The accuracy of the results of these methods is largely dependent on the quality of the scenarios used for the SA evaluation as these are scenario-based methods [8].…”