Robust experimental designs for model calibration

Abstract

A computer model can be used for predicting an output only after specifying the values of some unknown physical constants known as calibration parameters. The unknown calibration parameters can be estimated from real data by conducting physical experiments. This paper presents an approach to optimally design such a physical experiment. The problem of optimally designing a physical experiment, using a computer model, is similar to the problem of finding an optimal design for fitting nonlinear models. However, the problem is more challenging than the existing work on nonlinear optimal design because of the possibility of model discrepancy, that is, the computer model may not be an accurate representation of the true underlying model. Therefore, we propose an optimal design approach that is robust to potential model discrepancies. We show that our designs are better than the commonly used physical experimental designs that do not make use of the information contained in the computer model and other nonlinear optimal designs that ignore potential model discrepancies. We illustrate our approach using a toy example and a real example from industry.

Design choice and machine learning model performances

“…32 In this article we mainly leverage one of the advantages of the updated MAXPRO designs: if a limited number of levels is identified, the continuous factors can be restricted to the discrete numeric case, meaning the MAXPRO designs are also applicable in physical experimentation. 33 In our view, and for the scope of the present paper, this makes MAXPRO designs with discrete numeric factors (MAXPRO_dis) hybrid designs: while they are constructed to be essentially space-filling (although only on a limited number of levels), the limited number of levels makes them competitive with RSM designs for physical experimentation. 33 To our knowledge, this is the only class of designs of the space-filling type that can actually be applied to physical experimentation 33 by taking a limited number of levels for each continuous factor.…”

“…33 In our view, and for the scope of the present paper, this makes MAXPRO designs with discrete numeric factors (MAXPRO_dis) hybrid designs: while they are constructed to be essentially space-filling (although only on a limited number of levels), the limited number of levels makes them competitive with RSM designs for physical experimentation. 33 To our knowledge, this is the only class of designs of the space-filling type that can actually be applied to physical experimentation 33 by taking a limited number of levels for each continuous factor. The MAXPRO_dis design selected for this study counts six equally-spaced levels for each factor in the test functions.…”

Design choice and machine learning model performances

“…number of flutes in a solid end milling process) 32 . In this article we mainly leverage one of the advantages of the updated MAXPRO designs: if a limited number of levels is identified, the continuous factors can be restricted to the discrete numeric case, meaning the MAXPRO designs are also applicable in physical experimentation 33 . In our view, and for the scope of the present paper, this makes MAXPRO designs with discrete numeric factors (MAXPRO dis) hybrid designs: while they are constructed to be essentially space-filling (although only on a limited number of levels), the limited number of levels makes them competitive with RSM designs for physical experimentation 33 .…”

“…In our view, and for the scope of the present paper, this makes MAXPRO designs with discrete numeric factors (MAXPRO dis) hybrid designs: while they are constructed to be essentially space-filling (although only on a limited number of levels), the limited number of levels makes them competitive with RSM designs for physical experimentation 33 . To our knowledge, this is the only class of designs of the space-filling type that can actually be applied to physical experimentation 33 by taking a limited number of levels for each continuous factor. The MAXPRO dis design selected for this study counts 6 equally-spaced levels for each factor in the test functions.…”

Input‐response space‐filling designs

“…Kennedy and O'Hagan 3 and Leatherman et al 4 propose designs with the goal of optimizing the quality of prediction for the updated model that leverages the newly collected data with an existing understanding of the relationship between inputs and response. Krishna et al 5 and Huang 6 also describe methods for the strategic placement of experimental runs in the input space to optimize the prediction capability of the estimated models that take advantage of current understanding of the response surfaces. While also wanting to utilize what is already known about the existing model form and parameter estimates, we propose a design construction method with a different purpose.…”