A statistical method for tuning a computer code to a data base

Summary. We consider prediction and uncertainty analysis for systems which are approximated using complex mathematical models. Such models, implemented as computer codes, are often generic in the sense that by a suitable choice of some of the model's input parameters the code can be used to predict the behaviour of the system in a variety of speci®c applications. However, in any speci®c application the values of necessary parameters may be unknown. In this case, physical observations of the system in the speci®c context are used to learn about the unknown parameters. The process of ®tting the model to the observed data by adjusting the parameters is known as calibration. Calibration is typically effected by ad hoc ®tting, and after calibration the model is used, with the ®tted input values, to predict the future behaviour of the system. We present a Bayesian calibration technique which improves on this traditional approach in two respects. First, the predictions allow for all sources of uncertainty, including the remaining uncertainty over the ®tted parameters. Second, they attempt to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best-®tting parameter values. The method is illustrated by using data from a nuclear radiation release at Tomsk, and from a more complex simulated nuclear accident exercise.

show abstract

“…Their method does not account for remaining parameter uncertainty at the prediction stage. See also Cox et al (1996).…”

Section: Statistical Methods and Previous Workmentioning

confidence: 99%

Bayesian Calibration of Computer Models

Kennedy

O'Hagan

2001

Journal of the Royal Statistical Society Series B: Statistical Methodology

3,410

3,119

View full text Add to dashboard Cite

show abstract

“…Cox et al [7] proposed the approximate non-linear least-squares (ANLS) method that fits a GP model to the data from the computer experiment and then minimizes the discrepancy between the observed data from the physical experiment and the prediction from the GP model with respect to the tuning parameters. If we denote the output from the physical experiment by y p and the output from the computer experiment by y c then the ANLS method minimizes…”

Section: Introductionmentioning

confidence: 99%

“…Using model (2), they proposed that θ can alternatively be estimated using a maximum likelihood method. [7] Loeppky et al [8] also proposed a method for tuning based on the Maximum Likelihood approach. Recently, Han et al [9] proposed a Bayesian method to simultaneously estimate tuning and calibration parameters of a computer model (for more on calibration parameter estimation see Campbell, [10] Han et al, [9] Higdon et al, [11,12] Kennedy and O'Hagan, [13] Loeppky et al [8]) All these methods, however, use a space filling design for the computer experiment.…”

Section: Introductionmentioning

confidence: 99%

Sequential tuning of complex computer models

Kumar

2013

Journal of Statistical Computation and Simulation

View full text Add to dashboard Cite

We propose a method that uses a sequential design instead of a space filling design for estimating tuning parameters of a complex computer model. The goal is to bring the computer model output closer to the real system output. The method fits separate Gaussian process (GP) models to the available data from the physical experiment and the computer experiment and minimizes the discrepancy between the predictions from the GP models to obtain estimates of the tuning parameters. A criterion based on the discrepancy between the predictions from the two GP models and the standard error of prediction for the computer experiment output is then used to obtain a design point for the next run of the computer experiment. The tuning parameters are re-estimated using the augmented data set. The steps are repeated until the budget for the computer experiment data is exhausted. Simulation studies show that the proposed method performs better in bringing a computer model closer to the real system than methods that use a space filling design.

show abstract

“…Among the many possible combinations of the β's and θ's, we consider the following four models as basic ones (Cox et al, 2001): Here the "common θ" means that d number of θ's are forced to be a common θ c such that…”

Section: Existing Model Selection Algorithmmentioning

confidence: 99%

Model selection algorithm in Gaussian process regression for computer experiments

Lee

Park

2017

CSAM

Self Cite

View full text Add to dashboard Cite

The model in our approach assumes that computer responses are a realization of a Gaussian processes superimposed on a regression model called a Gaussian process regression model (GPRM). Selecting a subset of variables or building a good reduced model in classical regression is an important process to identify variables influential to responses and for further analysis such as prediction or classification. One reason to select some variables in the prediction aspect is to prevent the over-fitting or under-fitting to data. The same reasoning and approach can be applicable to GPRM. However, only a few works on the variable selection in GPRM were done. In this paper, we propose a new algorithm to build a good prediction model among some GPRMs. It is a post-work of the algorithm that includes the Welch method suggested by previous researchers. The proposed algorithms select some non-zero regression coefficients (β's) using forward and backward methods along with the Lasso guided approach. During this process, the fixed were covariance parameters (θ's) that were pre-selected by the Welch algorithm. We illustrated the superiority of our proposed models over the Welch method and non-selection models using four test functions and one real data example. Future extensions are also discussed.

show abstract

A statistical method for tuning a computer code to a data base

Cited by 51 publications

References 10 publications

Bayesian Calibration of Computer Models

Bayesian Calibration of Computer Models

Sequential tuning of complex computer models

Model selection algorithm in Gaussian process regression for computer experiments

Contact Info

Product

Resources

About