The impact of system representation choices on architecting insights

Abstract

Systems engineers regularly rely on analysis of early design artifacts like system architecture representations to predict system performance, lifecycle costs, and development schedules, and to support design decision‐making. Recent recognition of challenges in this type of measurement has led to a heightened focus on developing better metrics. Less attention has been paid to the system representations upon which all subsequent analysis is performed. With this study, we demonstrate that choices about how to represent the system can explain variation in measurement, even holding metrics constant. This is important because most of these representation choices remain unarticulated in current practice. To do this, we conduct a controlled experiment where we experimentally manipulated the Design Structure Matrix (DSM) architecture representation of nine crowdsourced robotic arm designs and compared the value and relative ranks of their modularity and complexity. We found statistically significant changes in both value and rank, attributable to differences in choices in the system representation. The direction and magnitude of these changes also differed across modularity and complexity. In addition, some underlying designs seemed to be more robust to representation changes. This suggests an interaction between representation, design, and lifecycle properties. These results emphasize the importance of developing standard guidelines for how to represent system architectures and better documenting their use.