Evaluating approaches for the next generation of difficulty and complexity assessment tools

Harvey

Engineering Management Journal

et al. 2021

As modern engineered systems become ever more connected and interdependent there is an increasing need to evaluate their complexity. However, evaluating system complexity is challenging due to the complicated conceptual landscape of competing definitions of the term complexity itself, and the range of perspectives that can be taken on what constitutes the System of Interest. This paper attempts to overcome these hurdles through introducing a Complexity Register with which to build and record a shared understanding of system complexity for key stakeholders. In order to overcome current challenges in evaluating the complexity of an engineering system, the Complexity Register encourages personnel to adopt a broad range of perspectives on the potential issues, impacts and mitigations to manage system complexity. The formulation of the Complexity Register is informed by design principles derived from a case study analysis of a system complexity evaluation tool. The Complexity Register should enable more effective shared understanding by encouraging collaboration that makes explicit the multiple viewpoints taken when evaluating system complexity and promoting continued re-evaluation throughout a system or project lifecycle.

Section: Complexity Assessment Toolsmentioning

confidence: 99%

The Complexity Register: A Collaborative Tool for System Complexity Evaluation

Harvey

Engineering Management Journal

et al. 2021

“…Complexity assessment tools and complexity categorization frameworks are discussed in more detail in Refs. 69, 70.…”

Section: Literature Reviewmentioning

confidence: 99%

Evaluating the complexity of engineered systems: A framework informed by a user case study

Bullock

2020

Systems Engineering

Evaluating the complexity of an engineered system is challenging for any organization, even more so when operating in a System‐of‐Systems (SoS) context. Here, we analyze one particular decision support tool as an illustratory case study. This tool has been used for several years by Thales Group to evaluate system complexity across a variety of industrial engineering projects. The case study is informed by analysis of semistructured interviews with systems engineering experts within the Thales Group. This analysis reveals a number of positive and negative aspects of (i) the tool itself and (ii) the way in which the tool is embedded operationally within the wider organization. While the first set of issues may be solved by making improvements to the tool itself, informed by further comparative analysis and growing literature on complexity evaluation, the second “embedding challenge” is distinct, seemingly receiving less attention in the literature. In this paper, we focus on addressing this embedding challenge, by introducing a complexity evaluation framework, designed according to a set of principles derived from the case study analysis; namely that any effective complexity evaluation activity should feature collaborative effort toward building an evaluation informed by a shared understanding of contextually relevant complexity factors, iterative (re‐)evaluation over the course of a project, and progressive refinement of the complexity evaluation tools and processes themselves through linking project evaluations to project outcomes via a wider organizational learning cycle. The paper concludes by considering next steps including the challenge of assuring that such a framework is being implemented effectively.

“…Moreover, what counts as a reasonable approach to defining system complexity depends on what type of system of interest (SoI) is being considered; is it limited to the technical system(s) being developed and deployed , or does it also include the systems of processes and resources that are involved in developing and deploying such technical systems ? 35 Is the project that strives to realize the system under consideration? 36–39 Does it include the processes of utilizing the system once deployed or the user's perceptions of how complex the system is (e.g., how familiar users of the system are with important features of the system)? 40,41 What is the boundary of the SoI; is it the physical context of the implemented system or does it also include the more extended strategic/business context? 42–46 While several approaches purport to provide a quantitative measure of the complexity of a system, they more realistically provide a quantitative measure of the complexity of a particular representation of a system (i.e., a particular view on the architecture of a system) 25,26,47–50 . A distinction is also required between the complexity of a representation of a system (i.e., the structural complexity of a system architecture) and the qualitative perceived and observer‐dependent complexity of the system 25,26,40 .…”

Section: Literature Reviewmentioning

confidence: 99%

Assaying the importance of system complexity for the systems engineering community

Sartor

et al. 2020

Systems Engineering

How should organizations approach the evaluation of system complexity at the early stages of system design in order to inform decision making? Since system complexity can be understood and approached in several different ways, such evaluation is challenging. In this study, we define the term “system complexity factors” to refer to a range of different aspects of system complexity that may contribute differentially to systems engineering outcomes. Views on the absolute and relative importance of these factors for early–life cycle system evaluation are collected and analyzed using a qualitative questionnaire of International Council on Systems Engineers (INCOSE) members (n = 55). We identified and described the following trends in the data: there is little between‐participant agreement on the relative importance of system complexity factors, even for participants with a shared background and role; participants tend to be internally consistent in their ratings of the relative importance of system complexity factors. Given the lack of alignment on the relative importance of system complexity factors, we argue that successful evaluation of system complexity can be better ensured by explicit determination and discussion of the (possibly implicit) perspective(s) on system complexity that are being taken.