Design and Analysis of Simulation Experiments: Tutorial

Kleijnen, J.P.C.

doi:10.2139/ssrn.2941492

Cited by 13 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The variety of experimental designs and modelling techniques for computer experiments is abundant and, just like in physical experiments, the selection of experimental design and data analysis technique depend on experimentation purpose (screening or modelling). Garud et al [82] and Kleijnen [84] provided general reviews on experimental designs and modelling techniques for computer experiments. The former summarizes classic linear regression metamodels, including polynomials, and their designs, and explains how sequential bifurcation can screen hundreds of variables.…”

Section: Selection Of Experimental Design and Analysis Methodsmentioning

confidence: 99%

“…This is considered the most efficient and effective method for screening in simulation experiments [88]. This is particularly true when hundreds of factors are involved, the direction of influence of each factor is known and nonnegative, and high experimental costs are expected if RSM designs (classical designs for physical experiments) are adopted [84,99]. As these tools mature, the need for practitioner-oriented guidance and easy-to-use free or commercial software is clear [83,94], because there is no evidence that computer experiments analysis is included in the curricular program of most engineering courses as a curricular unit, such as classical experimental design, which is not considered yet [43].…”

Section: Selection Of Experimental Design and Analysis Methodsmentioning

confidence: 99%

“…This has spawned a great deal of research in the last three decades and sparked the development of new statistical methods for designing and analysing the called simulation or computer experiments. Simulators are becoming increasingly prevalent surrogates for physical experiments, but practitioners often treat their experiments with simulators (or simulation models) very unprofessionally [83,84]. Trial-and-error and brute force (massive computation) practices are not recommended and, in most situations, must be not used, because they may neither uncover all the input variables with practical effect on the response nor the best functional relationship between input variables and response, and therefore, the most favourable settings for input variables.…”

Section: Doe and Simulation Experimentsmentioning

confidence: 99%

“…With this (small) set of variables, a two-level fractional factorial design (2 k-p design with k variables or factors and p independent generators) was selected. This design type is often used for screening in physical and simulation experiments when the number of variables is small (say up to about 15 to 20 variables, as it is common in physical experiments) due to their efficiency, effectiveness, and versatility for sequential experimentation [57,84]. In 2 k-p designs, each factor is tested at two levels and only a fraction (1/2 p ) of all factor-level combinations are run.…”

Section: Design and Analysis Of Experiments: Theoretical Framework Anmentioning

confidence: 99%

See 3 more Smart Citations

Energy-Efficiency Assessment and Improvement—Experiments and Analysis Methods

Costa

Fontes

2020

Sustainability

View full text Add to dashboard Cite

Some (non)manufacturing industries are becoming more energy efficient, but many of them are losing cost-effective energy-savings opportunities, namely, by lack of knowledge or underestimation of good engineering and management practices as well as guidance on techniques or tools for that purpose. This study points out that Design of Experiments is a tool that cannot be ignored by managers and other technical staff, namely, by those who have the responsibility to eliminate energy waste and promote energy-efficiency improvement in industry, mainly in energy-intensive manufacturing industries. A review on Design of Experiments for physical and simulation experiments, supported on carefully selected references, is provided, since process and product improvement at the design and manufacturing stages increasingly rely on virtual tests and digital simulations. However, the expense of running experiments in complex computer models is still a relevant issue, despite advances in computer hardware and software capabilities. Here, experiments were statistically designed, and several easy-to-implement yet effective data analysis methods were employed for identifying the variables that must be measured with more accurate devices and methods to better estimate the energy efficiency or improve it in a billets reheating furnace. A simulation model of this type of furnace was used to run the experiments and the results analysis shows that variables with practical effect on the furnace’s energy efficiency are the percentage of oxygen in the combustion gases, the fuel flow in the burners, and the combustion air temperature.

show abstract

Section: Selection Of Experimental Design and Analysis Methodsmentioning

confidence: 99%

Section: Selection Of Experimental Design and Analysis Methodsmentioning

confidence: 99%

Section: Doe and Simulation Experimentsmentioning

confidence: 99%

Section: Design and Analysis Of Experiments: Theoretical Framework Anmentioning

confidence: 99%

See 2 more Smart Citations

Energy-Efficiency Assessment and Improvement—Experiments and Analysis Methods

Costa

Fontes

2020

Sustainability

View full text Add to dashboard Cite

show abstract

“…We assume that the I/O data are noise-free (as in deterministic simulation or "computer experiments") and the Kriging model is valid, so Kriging is an exact interpolator; i.e., the Kriging predictions are exactly equal to the previously observed "old" outputs. Furthermore, we assume that the primary goal of Kriging is to predict the output for a '"new" input combination or "point"; related goals may be validation, sensitivity analysis, optimization, and uncertainty analysis, as discussed in Kleijnen (2017).…”

Section: Introductionmentioning

confidence: 99%

Prediction for Big Data Through Kriging: Small Sequential and One-Shot Designs

Kleijnen

Beers

2020

American Journal of Mathematical and Management Sciences

View full text Add to dashboard Cite

Kriging-or Gaussian process (GP) modeling-is an interpolation method assuming that the outputs (responses) are more correlated, as the inputs (explanatory or independent variables) are closer. Such a GP has unknown (hyper)parameters that are usually estimated through the maximum-likelihood method. Big data, however, make it problematic to compute these estimated parameters, and the corresponding Kriging predictor and its predictor variance. To solve this problem, some authors select a relatively small subset from the big set of previously observed "old" data. These selection methods are sequential, and they depend on the variance of the Kriging predictor; this variance requires a specific Kriging model and the estimation of its parameters. The resulting designs turn out to be "local"; i.e., most selected old input combinations are concentrated around the new combination to be predicted. We develop a simpler oneshot (fixed-sample, non-sequential) design; i.e., from the big data set we select a small subset with the nearest neighbors of the new combination. To compare our designs and the sequential designs empirically, we use the squared prediction errors, in several numerical experiments. These experiments show that our design may yield reasonable performance.

show abstract

Gradient‐based criteria for sequential experiment design

Erickson

Ankenman

Plumlee

et al. 2021

Quality & Reliability Eng

View full text Add to dashboard Cite

Computer experiments are often used as inexpensive alternatives to real-world experiments. Statistical metamodels of the computer model's input-output behavior can be constructed to serve as approximations of the response surface of the real-world system. The suitability of a metamodel depends in part on its intended use. While decision makers may want to understand the entire response surface, they may be particularly keen on finding interesting regions of the design space, such as where the gradient is steep. We present an adaptive, value-enhanced batch sequential algorithm that samples more heavily in such areas while still providing an understanding of the entire surface. The design points within each batch can be run in parallel to leverage modern multi-core computing assets. We illustrate our approach for deterministic computer models, but it has potential for stochastic simulation models as well.

show abstract

Design and Analysis of Simulation Experiments: Tutorial

Cited by 13 publications

References 23 publications

Energy-Efficiency Assessment and Improvement—Experiments and Analysis Methods

Energy-Efficiency Assessment and Improvement—Experiments and Analysis Methods

Prediction for Big Data Through Kriging: Small Sequential and One-Shot Designs

Gradient‐based criteria for sequential experiment design

Contact Info

Product

Resources

About