Is Verifying Frequently an Optimal Strategy? A Belief-Based Model of Verification

Salado

et al. 2020

Self Cite

Verification activities, such as inspection, testing, analysis, and demonstration, improve one's confidence in the system meeting the system requirements during the development process. Frequent verification is often advocated as a strategy that minimizes costs of rework over the entire design process, where frequent verification involves verifying after any change in the design. However, this strategy is yet to be validated. In this paper, we develop a belief‐based model of verification in systems design to determine the conditions under which frequent verification is an optimal strategy for a vertically integrated organization. Our model uses belief distributions to capture the organization's dynamic confidence in the system design meeting a requirement of interest during the development process. It also captures the organization's dynamic confidence in the correctness of its development activities (or design process) as a function of past verification activities and current system maturity. The analysis of our model shows that frequent verification is a cost‐minimizing strategy for any level of belief in satisfying the requirement only when the organization has high confidence in the correctness of its design activities and the expected cost to rework a faulty design is greater than the costs to set up the verification activities throughout the development process. Otherwise, strategies with infrequent verification are superior. Our work contributes to the growing body of literature on the theoretical foundations of systems engineering and engineering design and seeks to provide practitioners with a means to determine optimal verification strategies.

“…The results presented in Section 4.1 generalize our previous work 42 . In this section, we explicitly define the effect of

ε_{n, l}

and N on frequent verification being an optimal strategy.…”

Section: Discussionsupporting

confidence: 68%

Section: Background and Motivationmentioning

confidence: 62%

See 1 more Smart Citation

An evaluation of the optimality of frequent verification for vertically integrated systems

Salado

et al. 2020

Self Cite

“…Yet, majority of the literature on verification activities adopts the aleatory uncertainty approach 3,5,21–37 . The reader is referred to ref 38,39 . for a discussion on these works and the drawbacks of adopting the aleatory uncertainty approach to model systems engineering projects.…”

Section: Introductionmentioning

confidence: 99%

“…Recent works on verification in systems engineering have begun to explore the benefits of modeling the epistemic uncertainty in systems engineering projects with belief distributions 38–47 . In these works, belief distributions are used to model the subjective confidence of designers in the true state of the system design.…”

Section: Introductionmentioning

confidence: 99%

Optimal verification strategies in multi‐firm projects

Salado

Wernz

2022

Self Cite

Verification activities are intended to reduce the costs of system development by identifying design errors before deploying the system. However, subcontractors in multifirm projects are motivated to implement locally cost-effective verification strategies over verification strategies that benefit the main contractor. Incentivizing verification activities is one mechanism by which the contractor can motivate subcontractors to implement verification strategies desirable to the contractor. Prior work on mathematical models of verification in systems engineering has neither explored optimal verification strategies nor incentives in multi-firm projects. In this paper, we present a modeling concept for determining optimal verification strategies in multi-firm projects. Our models are belief-based, which means that contractors and subcontractors incorporate their at times limited knowledge about true verification state through a probabilistic assessment of possible states. We develop an initial two-level model, where one contractor directly works with multiple subcontractors at the next lower level. This model is then extended to a general network model with multiple, multilevel contractor-subcontractor relationship. We derive solution algorithms that characterize the optimal verification strategies and incentives for each of the firms. Our work contributes to the systems engineering literature by laying the foundation for the study of incentives as a mechanism to align verification activities in multi-firm systems engineering projects.

Optimal verification strategy for general development plans using a belief‐based approach

Girod

et al. 2023

Self Cite

Verification activities are intended to reduce the costs of system development by identifying design errors before deploying the system. However, subcontractors in multi‐firm projects are motivated to implement locally cost‐effective verification strategies over verification strategies that benefit the main contractor. Incentivizing verification activities is one mechanism by which the contractor can motivate subcontractors to implement verification strategies desirable to the contractor. In this paper, we present a belief‐based modeling concept for determining optimal verification strategies for general development plans. The results show that the optimal incentives are a function of the subordinate firm's beliefs and the influence exerted by the subordinate firm on the supervising firm with respect to verification activities.