How well can we predict changes at architecture design time?

Abstract

Two years ago, we analyzed the architecture of Sagitta 2000/SD, a large business information system being developed on behalf of Dutch Customs. We were in particular interested in assessing the capabilities of the system to accommodate future complex changes. We asked stakeholders to bring forward possible changes to the system, and next investigated how these changes would affect the software architecture. Since then, the system has been implemented and used, and actual modifications have been proposed and realized. We studied all 117 change requests submitted since our initial analysis. The present paper addresses how well we have been able to predict complex changes during our initial analysis, and how and to what extent the process to elicit and assess the impact of such changes might be improved. This study suggests that architecture analysis can be improved if we explicitly challenge the initial requirements. The study also hints at some fundamental limitations of this type of analysis:(1) fundamental modifiability-related decisions need not be visible in the documentation available, (2) the actual evolution of a system remains, to a large extent, unpredictable and (3) some changes concern complex components, and this complexity might not be known at the architecture level, and/or be unavoidable.

Measuring Architecture Sustainability

“…The probabilities and impacts of some scenarios needed to be updated due to changing assumptions. Contrary to former studies [11], we did not identify new likely evolution scenarios. While some components had been replaced or extended to cover new features, the overall architecture remained stable.…”

Sustainability evaluation of software architectures

“…There is no documentation template provided. Determining ripple effects is explicitly foreseen but relies on the experience of the involved architects and developers, which can be misleading [42]. There is no support for analysing variation and extension points and there is no guidance on how to improve the architecture after evaluation.…”

“…It offers some helpful techniques for change scenario elicitation but still relies heavily on the experience of the involved stakeholders as there is no guidance on how to improve the architecture. Some experience reports with ALMA offer interesting insights to sustainability evaluation [41,42]: During change scenario elicitation architects are biased towards the scenarios they already had in mind when designing the architecture. Determining the components affected by a change scenario is often straight-forward, but determining ripple effects is not.…”

Software architecture evolution through evolvability analysis

“…Another example is related to scenario-based analysis methods. Most scenario-based software architecture analysis methods have the strength of being able to concretize driving quality attribute requirements, but they also have a weakness of being optimistic in change scenario elicitation due to the unpredictable nature of changes as well as stakeholders' short horizon in foreseeing future changes [110]. Therefore, some architectural knowledge management approaches can be used to complement scenario-based methods and address this weakness through explicit representation of invariabilities to provide additional what-if scenarios.…”