Verification, Validation, and Accreditation

Petty, Mikel D.

doi:10.1002/9780470590621.ch10

Cited by 32 publications

(21 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Petty goes on to stipulate that, except for the simplest of models, it is largely incorrect to refer to complex models as “valid” or “validated.” More appropriately, validation is a qualified description of validity that is interpreted by its users as accurate enough to be useful within the parameters and context (bounds of validity) set forth in its design . We will return to Petty's three questions in Section by mapping them into the TRAM validation testing concepts and questions presented in this article (Table ).…”

Section: A Broad View Of Validationmentioning

confidence: 99%

Validation in the Absence of Observed Events

Lathrop¹,

Ezell

2015

Risk Analysis

View full text Add to dashboard Cite

This article addresses the problem of validating models in the absence of observed events, in the area of weapons of mass destruction terrorism risk assessment. We address that problem with a broadened definition of "validation," based on stepping "up" a level to considering the reason why decisionmakers seek validation, and from that basis redefine validation as testing how well the model can advise decisionmakers in terrorism risk management decisions. We develop that into two conditions: validation must be based on cues available in the observable world; and it must focus on what can be done to affect that observable world, i.e., risk management. That leads to two foci: (1) the real-world risk generating process, and (2) best use of available data. Based on our experience with nine WMD terrorism risk assessment models, we then describe three best use of available data pitfalls: SME confidence bias, lack of SME cross-referencing, and problematic initiation rates. Those two foci and three pitfalls provide a basis from which we define validation in this context in terms of four tests--Does the model: … capture initiation? … capture the sequence of events by which attack scenarios unfold? … consider unanticipated scenarios? … consider alternative causal chains? Finally, we corroborate our approach against three validation tests from the DOD literature: Is the model a correct representation of the process to be simulated? To what degree are the model results comparable to the real world? Over what range of inputs are the model results useful?

show abstract

Section: A Broad View Of Validationmentioning

confidence: 99%

Validation in the Absence of Observed Events

Lathrop¹,

Ezell

2015

Risk Analysis

View full text Add to dashboard Cite

show abstract

“…Accreditation in various definitions, centered on the review process (peer review system, expertisebased assessment, evaluation and assessment), using agreed standard instruments and references in order to maintain quality and to evaluate that the accredited entity has a commitment always to achieve high quality, and still consistently committed to implementing a system of continuous quality improvements (Management, 2009;Petty, 2010). In the context of the study program, it generally relies heavily on an evaluation system using the peer review system, reviewers, assessors, or auditors.…”

Section: Introductionmentioning

confidence: 99%

Analysis of the accreditation grade of study programs in Indonesia higher education

Sutopo¹,

Sugiyono²,

Setiadi³

2019

Proceedings of the 1st International Conference on Education Social Sciences and Humanities (ICESSHum 2019)

View full text Add to dashboard Cite

This study aims to investigate the accreditation grade of college study program in Indonesia. The study was based on seven standards of accreditation instruments of higher education. It compares the grade of institutional groups, how the grades varies among study programs in Java and Outside Java, and compares the grades among regions of private colleges (Kopertis). The analytical method used in this study is evaluation research. The technique for data collection is documentation of accreditation results conducted by assessors of study programs in 2015. The results of the analysis show that: the overall lowest grade is Standard 7 with the average of 2.62 out of 4. The averages results of standard 6, 4, and 3 are 2.74, 2.78, and 2.83 respectively. The comparisons of the accreditation grades between Java and outside Java study programs are: 1678 and 268 study programs that received A grade, 4627 to 3422 received B grade, and 3901 to 8641 that received C grade, respectively. The top five provinces in which the colleges received A grades are Yogyakarta, Jakarta, West Java, East Java, and Central Java provinces with the percentages of 32, 18, 18, 13 and 12, respectively. In contrast, the provinces in which there are no college which received A grades are Gorontalo,

show abstract

“…Model verification is a process that determines whether the programming implementation of the conceptual model is correct, i.e. “is the model made right” (Petty, 2010, p. 332). This process includes debugging the software, testing the logic, looking for incorrect implementation of conceptual models, and verifying calculations (Louie and Carley, 2008; Wang and Lehmann, 2007).…”

Section: Introductionmentioning

confidence: 99%

“…This process includes debugging the software, testing the logic, looking for incorrect implementation of conceptual models, and verifying calculations (Louie and Carley, 2008; Wang and Lehmann, 2007). Model validation answers the question “Was the right model made?” (Petty, 2010, p. 332) and is a process that determines whether the conceptual model is a reasonably accurate representation of the real world and produces outcomes consistent with the real world (it behaves as expected) (Marks, 2007; Midgley et al, 2007; Rand and Rust, 2011; Windrum et al, 2007). During model validation processes parameters are calibrated/estimated to improve the model alignment with real-world data and ultimately produce evidence that the model is sufficiently accurate for the planned application (accreditation) – Petty (2010).…”

Section: Introductionmentioning

confidence: 99%

“…Model validation answers the question “Was the right model made?” (Petty, 2010, p. 332) and is a process that determines whether the conceptual model is a reasonably accurate representation of the real world and produces outcomes consistent with the real world (it behaves as expected) (Marks, 2007; Midgley et al, 2007; Rand and Rust, 2011; Windrum et al, 2007). During model validation processes parameters are calibrated/estimated to improve the model alignment with real-world data and ultimately produce evidence that the model is sufficiently accurate for the planned application (accreditation) – Petty (2010). When empirical evidence is lacking, a systematic exploration of different parameters’ space provides greater assurance in the validity of the parameter estimates and ranges (Saltelli et al, 2008).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Rethinking Validation: Efficient Search of the Space of Parameters for an Agent-Based Model

Olaru

Purchase

2014

Australasian Marketing Journal

View full text Add to dashboard Cite

This paper shows how sensitivity analysis can be used as part of model verification and validation Sensitivity analysis provides insights on where future data validation processes should focus and which inputs may be considered for model reduction. We compared two approaches, one using a systematic variation of parameter values, another using an optimised algorithm to make more efficient the search of their space. Analysis was conducted on an agent-based model that explores the emergence of innovation within business networks, where successful innovation is considered an increase in knowledge and financial resources within the network. The two sensitivity analysis approaches differed both on their time efficiency and on the type of information provided. While the systematic individual sensitivity analysis assisted us in identifying inputs with substantial impact upon the results and suggest solutions for model simplification, the optimised search provided insights on the network resources likely to achieve higher levels of innovation. Genetic algorithms found parameter values that produced different results in the agent-based model.

show abstract

Verification, Validation, and Accreditation

Cited by 32 publications

References 22 publications

Validation in the Absence of Observed Events

Validation in the Absence of Observed Events

Analysis of the accreditation grade of study programs in Indonesia higher education

Rethinking Validation: Efficient Search of the Space of Parameters for an Agent-Based Model

Contact Info

Product

Resources

About