A comparison of the robustness of reliability growth assessment techniques

Wayne, Martin; Ellner, Paul M.

doi:10.1109/rams.2010.5448064

Cited by 5 publications

(10 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 show that the proposed model outperforms both the AMPM and Crow Extended models in terms of accuracy. Additional simulation comparisons over a broader set of test cases are presented in [7], and the results indicate that AMPM outperforms the Crow Extended Model with respect to accuracy and relative error distributions. By extension, the proposed approach should therefore perform well over a broad range of cases.…”

Section: Simulation Performancementioning

confidence: 99%

“…This approach allows for more flexibility in the projection framework, as the models can project the anticipated improvement that will result from corrective actions that may not yet be implemented. The use of FEFs has also been shown to provide very reasonable estimates of reliability improvement when they are assigned correctly [7]. Reliability growth planning models are then usually just transformed versions of the assessment models that can be used to plan an appropriate reliability growth program.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Bayesian Model for Complex System Reliability Growth Under Arbitrary Corrective Actions

Wayne

Modarres²

2015

IEEE Trans. Rel.

Self Cite

View full text Add to dashboard Cite

This paper presents a new method for projecting the reliability growth of a complex continuously operating system. The model allows for arbitrary corrective action strategies, and it differs from other models of this type by using all available data rather than failure mode first occurrence times only. It also differs from other reliability growth projection models in that it provides a complete inference framework via the posterior distribution on the system failure intensity. A unique feature of this approach relative to other Bayesian techniques is the analytic expression for the failure intensity contribution from unobserved failure modes. Expressions for estimating the initial failure intensity, growth potential failure intensity, and the cumulative number of failure modes expected in future testing are also developed. Extensions to the basic framework are also developed. The first accounts for multiple systems under test, and the second develops the posterior distribution while allowing for uncertainty on the Fix Effectiveness Factor values that are assessed. Two separate goodness-of-fit procedures are presented for assessing the appropriateness of the underlying model assumptions.Index Terms-Bayesian reliability, fix effectiveness, reliability growth, reliability growth projection.

show abstract

Section: Simulation Performancementioning

confidence: 99%

mentioning

confidence: 99%

A Bayesian Model for Complex System Reliability Growth Under Arbitrary Corrective Actions

Wayne

Modarres²

2015

IEEE Trans. Rel.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sarhan, Guess, and Usher (2007) modeled component failure probabilities as iid beta random variables for a multicomponent system in the presence of dependent masked system life test data. A reliability projection approach for continuous use systems that utilizes failure mode count data and individual failure mode FEFs in a likelihood was presented by Wayne and Ellner (2010). The discussants correctly point out that k is unknown and is difficult to statistically estimate.…”

Section: Gaver and Jacobsmentioning

confidence: 99%

“…In a simulation study conducted with a continuous projection model (Wayne and Ellner 2010) the limiting estimates were quite close to the finite k based estimates when k was at least twice m. Not being able to graphically distinguish an estimated metric curve based on k from the corresponding curve of limiting metric estimates as functions of the trial number over the dataset test period is a strong indication that the data cannot be relied on to provide an accurate estimate of the total number of failure modes or the number of unobserved failure modes. 52, NO.…”

Section: Wilson and Anderson-cookmentioning

confidence: 99%

Reliability Growth Management Metrics and Statistical Methods for Discrete-Use Systems

2010

View full text Add to dashboard Cite

In this article we present new methodology for analyzing reliability growth of discrete-use systems (i.e., systems whose test duration is measured in terms of discrete trials, shots, or demands). The methodology is applicable to the case where corrective actions are applied to prototypes anytime after associated failure modes are first discovered. Thus, the system configuration need not be constant. The methodology consists of several model equations for estimating: system reliability; the expected number of failure modes observed during testing; the probability of failure due to a new failure mode, and the portion of system unreliability associated with repeat, or known, failure modes. These model equations are used to: (1) estimate the initial and projected reliability as well as the reliability growth potential; (2) address model goodnessof-fit concerns; (3) quantify programmatic risk; and (4) assess reliability maturity of discrete-use systems undergoing development. Statistical procedures for point estimation, confidence interval construction, and goodness-of-fit testing are also given. In particular, a new likelihood function (and associated maximum likelihood procedure) is derived to estimate model parameters, that is, the shape parameters of the beta distribution. An application to a missile program is given to illustrate the utility of the presented approach. Supplemental materials for this article are available on the Technometrics website.

show abstract

“…However, such an approach would employ a likelihood function that contained failure mode fix effectiveness factors that are typically assessed by engineering judgment. A reliability projection approach for continuous use systems that utilizes failure mode count data and individual failure mode FEFs in a likelihood was presented by Wayne and Ellner (2010). Problematic technical issues and potential benefits of such an approach are discussed in our rejoinder to Sen and Fries.…”

mentioning

confidence: 99%

Rejoinder

Hall¹,

Ellner²,

Mosleh³

2010

Technometrics

View full text Add to dashboard Cite

GAVER AND JACOBSWe would first like to thank Gaver and Jacobs for their thoughtful discussion. With respect to the issues raised, we regard the potential masking problem to be of greatest concern. Viewing a discrete-use mission "trial" as an ordered sequence of stages as they do clearly points to the problem that the occurrence of a failure mode at stage i can preclude a potential failure mode associated with a later stage from occurring on the trial. Such preclusion effects violate the model's failure mode occurrence independence assumptions. However, if the graphical and statistical goodness-of-fit procedures do not provide evidence that the observed failure mode first occurrence trial pattern is inconsistent with the model's expected number of failure modes metric, we would deem such preclusion effects to be insignificant for the observed test data. On the contrary, if there were statistical evidence against the model, alternate projection assessment techniques will have to be employed. Such techniques that provide additional (and sometimes needed) model fidelity not only increase model complexity, but much more importantly, typically increase the number of model parameters that must be assessed. This in turn can potentially degrade the accuracy of the reliability projection. Thus, we would recommend applying a projection model that is as parsimonious as possible with respect to model parameters, subject to the condition that the observed data is consistent with the model. The discussants write: "the first model" where "no fixes occur until T tests/trials elapse, during which no repair/redesign is made. (The second model does permit sequential fixes.)" Our article addresses what the discussants refer to as the "second model." This model accommodates a corrective action strategy that incorporates fixes to failure modes at any time after the failure mode has been observed in test. Under such a scenario, the initial probability of occurrence of an observed failure mode may not be constant over the T trials. Thus it may be difficult to obtain an accurate statistical estimate of the initial mode probability of occurrence from the test data. Such an estimate is desired if one wishes to utilize individual mode FEFs to reduce the initial probabilities of occurrence of observed failure modes to which corrective actions have been applied during the test period. This consideration motivated using the average of the assessed FEFs for the observed failure modes.

show abstract

A comparison of the robustness of reliability growth assessment techniques

Cited by 5 publications

References 4 publications

A Bayesian Model for Complex System Reliability Growth Under Arbitrary Corrective Actions

A Bayesian Model for Complex System Reliability Growth Under Arbitrary Corrective Actions

Reliability Growth Management Metrics and Statistical Methods for Discrete-Use Systems

Rejoinder

Contact Info

Product

Resources

About