Traditional document‐based practices in systems engineering are being transitioned to model‐based ones. Adoption of model‐based systems engineering (MBSE) continues to grow in industry and government, and MBSE continues to be a major research theme in the systems engineering community. In fact, MBSE remains a central element in the International Council on Systems Engineering (INCOSE)’s vision for 2025. Examining systems engineering literature, this paper presents an assessment of the extent to which benefits and value of MBSE are supported by empirical evidence. A systematic review of research and practice papers in major systems engineering archival journals and conference proceedings was conducted. Evidence was categorized in four types, two of which inductively emerged from the results: measured, observed (without a formal measurement process), perceived (claimed without evidence), and backed by other references. Results indicate that two thirds of claimed MBSE benefits are only supported by perceived evidence, while only two papers reported measured evidence. The aggregate assessment presented in this paper indicates that claims about the value and benefits of MBSE are mainly based on expectation. We argue that evidence supporting the value and benefits of MBSE remains inconclusive.

Model-based systems engineering (MBSE) is an increasingly accepted practice in the Systems Engineering (SE) community, however, little has been done to empirically show that MBSE provides value. Furthermore, as the industry continues in the direction of digital transformation, MBSE will become a critical component of the larger Digital Engineering (DE) approach. This paper presents a measurement framework for selecting and developing appropriate metrics to assess the value/benefits of MBSE and subsequently DE. Utilizing expected benefits identified in a review of MBSE literature, a causal map was hypothesized to show how expected benefits (potential metrics) influence and relate to each other. This was done in order to systematically determine which benefits would be the most impactful to measure. The hypothesized causal model was presented for feedback to subject-matter experts from a working group developing the first DE measurement framework. This group is a joint effort with industry, academia, and the USA government to develop DE metric standards. Once the causal map was finalized, a case study was used to partially validate the causal model.Based on the causal map and subsequent analysis, we can recommend the first metrics to be employed for DE/MBSE based on the most influential nodes of the causal model.The potential metric candidates include: system quality, defects, time, rework, ease of making changes, system understanding, Effort, accessibility of information, collaboration, project methods/processes, and use of DE/MBSE tools. We believe a concerted effort across the industry to focus on measuring these variables is the most effective way to establish proof of the value of MBSE and DE.

Those looking to advocate for Model‐Based Systems Engineering (MBSE) in the Systems Engineering field often turn to more established fields that have made a similar transition to models to assure others it will be beneficial. One practice that is often compared to MBSE is Computer‐Aided Design (CAD) from the field of mechanical engineering. However, the adequacy of this comparison is challenged upon a side‐by‐side examination of what MBSE and CAD are. Based on the established definitions, it is evident that while CAD can be considered a method of mechanical drawing, MBSE cannot be described as only a method. MBSE is more than installing and utilizing software, it is a process in and of itself. Comparing MBSE to CAD runs the risk of oversimplifying MBSE and setting up expectations that may not be met. Therefore, while CAD may represent a similar paradigm shift as MBSE in terms of digitalization, it may not serve as an adequate paradigm in terms of adoption and benefits. In this paper, we characterize and compare CAD and MBSE and identify the similarities and differences between them. We use the resulting insights to level the expectations of adopting and using MBSE.

Digital Engineering and Model-Based Engineering continue to grow in adoption across the systems engineering community. However, programs and enterprises still struggle to quantify the value of this digital transformation. Research conducted by the Systems Engineering Research Center in collaboration with a government/industry Digital Engineering Measures Working Group is creating the first formal measurement framework for this transformation. This article describes the research, formation of a causal measurement model, and initial specification of candidate measures.

No abstract