Verification and Validation Issues in a Generic Model of Electro-Optic Sensor Systems

Abstract

In general, questions of model credibility introduce more problems in a generic model than they do in models developed for one specific application since a generic formulation must allow for many applications of the model. This paper addresses the issues of model testing, verification, and validation for a generic electro-optic sensor system model. A structural approach to testing, verification, and validation is proposed that builds increasing confidence through bottomup testing, structured verification procedures, and carefully selected validation metrics. These metrics are based on a geometrical view of model outputs that may be compared with measurements using qualitative methods or quantitative approaches involving image processing, artificial neural networks, or fuzzy pattern recognition. The advantage over traditional validation methods is most marked in the case of complex models with many key quantities where it not only provides insight about the validity but also about sensitivities. These validation tools have been applied, in conjunction with more traditional metrics, to the testing, verification, and validation of the generic model configured as a thermal imager system.

Issues of model quality and the validation of dynamic models

“…The distortion of the model polygon following a specifi c change in the model provides a clear indication of sensitivities and interactions, as discussed in papers by Smith et al . ( [16] and [17]). Shape-processed visualisation methods such as these lend themselves to image processing techniques for quantifi cation and a number of approaches have been considered (see [18] and [19]).…”

“…Polar diagrams such as these have been used successfully in the context of model testing for electro-optic sensor models ( [16] and [17]) and have been considered in the context of fault detection applications as well as in other model-testing situations ( [18] and [19]). Although they have been developed independently for model validation purposes, these diagrams have many features that are similar to those of the Kiviat diagrams ( [17] and [19]), which are applied in the software engineering fi eld for visualisation of metrics in computer software and hardware performance evaluation.…”

“…Although they have been developed independently for model validation purposes, these diagrams have many features that are similar to those of the Kiviat diagrams ( [17] and [19]), which are applied in the software engineering fi eld for visualisation of metrics in computer software and hardware performance evaluation.…”

Fitness-for-Purpose: Towards Understanding Limitations of Simulation Models for Engineering Systems Design

“…These are very satisfactory if only one or two variables are being considered but are difficult to interpret if many variables need to be considered. In such cases a type of polar diagram (similar to the Kiviat diagrams used in software engineering for visualisation of the results of software and hardware performance evaluations) can be very helpful [5], [6], [7]. Figure 1 shows an example of this type of polar diagram, as applied to the external validation process.…”

Mathematical and Computer Modeling of Electro-Optic Systems Using a Generic Modeling Approach

“…If, at any stage, it is found that the approach fails, then either a fundamental design flaw will have been exposed in the model or a limitation will have been found in the generic approach. Detailed issues of testing and validation of the generic model are discussed more extensively in an associated paper [12].…”