Design of computer simulation experiments for industrial systems

Burdick, Donald S.; Naylor, Thomas H.

doi:10.1145/355592.365616

Cited by 40 publications

(14 citation statements)

References 38 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approximations are also called surrogate and response surface models. Early work explored response surface models for simulations, under the name of "regression" or "experimental designs" (Walsh 1963, Burdick andNaylor 1966, Hunter andNaylor 1970). While these focused on stochastic, typically discrete-event, simulation (as are ours), metamodels are also used to approximate other kinds of simulation models (e.g., finite element, circuit, stochastic flow, and boundary element simulation models).…”

Section: Basics Of Metamodelingmentioning

confidence: 99%

Tutorial: Simulation metamodeling

Barton

2015

2015 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

The concept of a metamodel has been an important tool for simulation analysis for forty years. These models of simulation models have the advantage of faster execution, and they can (sometimes) provide insight on the nature of the simulation response as a function of design and input distribution parameters. This introductory tutorial will describe metamodeling uses and associated processes, survey commonly used metamodel types and associated experiment designs, and give a brief description of some recent developments and how they may affect future "mainstream" simulation metamodeling. 1 INTRODUCTIONDiscrete-event simulation models allow users to examine in relatively high fidelity the expected behavior of manufacturing, transportation, health care or other systems. The simulation model can be exercised at lower cost and with less risk than the real system. Further, the real system may not yet exist, so simulation provides performance prediction for the as-yet-to-be-constructed real system. Exercising such a model in an ad hoc way may not lead to general insights on the behavior of the real (either extant or imagined) system being modeled, however. Further, when the simulation model fidelity is high and the system is complex, the computational effort to simulate can be substantial. Simulationists then often fit a computationally efficient approximation to the simulation model: a metamodel. In this introductory tutorial we examine the role of metamodels and the metamodeling activity in simulation studies. Metamodeling permits special insights into system performance, and enables extensive exercise of the system under different design conditions at relatively modest computational cost. This is the first WSC introductory tutorial focused on metamodeling in at least seventeen years (Barton 1998). Closely related tutorials on the design of simulation experiments appear regularly, however. See for example the tutorials by Law (2014), Sanchez and Wan (2012), Barton (2010), and Kleijnen (2008). This is not to say that the field is not active: there have been many papers on new metamodeling methods and on new uses for simulation metamodels. At WSC'14 alone there were two sessions dedicated to metamodeling, and a total of ten papers across all sessions. And see the very good advanced tutorial by Staum (2009). This tutorial will provide an accessible view to metamodeling for the beginner. For that reason, open access materials receive preference: a reference to a Management Science paper may be omitted in favor of an earlier WSC publication -these can be freely accessed under the Archive link at the WSC home page (www.wintersim.org). Advanced methods will not be the focus. While this tutorial will discuss the design of simulation experiments, the emphasis will be on the historical context of metamodeling, metamodel types, and the ultimate uses of metamodels, with guidance on the overall metamodeling process.978-1-4673-9743-8/15/$31.00 ©2015 IEEE

show abstract

Section: Basics Of Metamodelingmentioning

confidence: 99%

Tutorial: Simulation metamodeling

Barton

2015

2015 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

show abstract

“…For the case where we fit a main effects model and the true system contains sccond-order effects, moments through order 3 must be equal and this is accomplished by minimization of F(x) in Equation ( [3][4][5][6][7][8][9][10][11][12] considering only the first four term.,;.…”

Section: (3-9)mentioning

confidence: 99%

“…The average mean square error is composed of two terms, the average variance (3)(4)(5)(6) and the average squared bias…”

mentioning

confidence: 99%

Methods for Factor Screening in Computer Simulation Experiments

Montgomery¹

1979

View full text Add to dashboard Cite

SUMMARY e use of a computer simulation model may be viewed as an experiment in which a set of k input variables are combined to produce at least one output or response variable.As in aiy experimental situation, the design of a computer simulation experiment is important. In general, not all k input variables or factors ill he equally important in their effect on the response variable(s). It is very con-non to find that only a subset, say g < k, of the original k factors are important in explaining the response. We usually do not know the value of g, or which g factors are important.The problem of experimentation and analysis to discover the size and composition of the subset of active factors g is called the factor screening problem.It is important to accurately identify the set of active factors.Failure to identify an active factor can result in serious bias in the analysis and conclusions drawn from the model, if that factor is subsequently ignored. Irregular fractional factorialsA logical screening strategy involving those designs is developed. The selection between designs Is based on consideration of the extent of aliasing of interactions and the severity of assumptions required to produce a unique analysis of the data. In particular, It is shown that group screening followed by the use of a 2 k-p fractional factorial deoign is often an optimal screening approach.Variance reduction methods for these designs are discussed. based on common and antithetic random number streams. Other problems discussed include the composition of the groups in group screening and selecting levels for negligible factors In sub-iequent experiments.* A second major type of screening study is the iundesigned case. These situations occur when there are data available from previous simulation experiments with the model, and decisions regarding the identification of active factors must be made using these data. It is unlikely that these runs will conform to any standard factor screening design. However, in these cases, the L. method of least squares can be used to fit an appropriate regression rModel to L the data, and factor screening decisions can often be made using this model. 1.TThe usual nonorthogonality of such undesigned data makes the interpretation of these midels difficult. Standardized regression coefficients can be used td simplify the interpretation, although this still does not Solve the problems created by a nonorthogonal data set. Several measures of nonorthogonality are introduced, including variance inflation factors and conditioning numbers, and the use of these measures in assessing the problems in interpreting individual regression coefficients is discussed. In cases of extreme nonorthogonality, parameter estimation methods other than least squares are recommended.The third type of factor screening study involves augmenting an available data set with a small number of new runs. The question of where these additional runs should be conducted is discussed. Two design augmentation methods are proposed, one based on...

show abstract

“…The reader should consult the references for a complete account of the theory of spectral analysis. 5 Spectral analysis considers data arranged in a series according to historical time. When one so plots the movement of many economic variables through time, he often observes a remarkable degree of smoothness in the curve; that is, although a variable assumes values that are at different times relatively high and low, its path, nevertheless, progresses evenly enough to suggest that the current value of that variable is related to its past values.…”

Section: Theory Of Spectral Analysismentioning

confidence: 99%

“…Complete exploitation of simulation experiments with econometric models implies a thorough analysis of the data so generated. Yet as Burdick and Naylor [5,31,32] have pointed out in recent articles, a preoccupation with model building among many econometricians simulating economic systems has unduly diverted attention from experimental design considerations in general, and output analysis in particular.…”

Section: Introductionmentioning

confidence: 99%

Spectral Analysis of Data Generated by Simulation Experiments with Econometric Models

Naylor¹,

Wertz²,

Wonnacott³

1969

Econometrica

Self Cite

View full text Add to dashboard Cite

This paper is concerned with the use of spectral analysis to analyze data generated by computer simulation experiments with models of economic systems. An example model serves to illustrate two different applications of spectral analysis. First, spectral analysis is used to construct confidence bands and to test hypotheses for the purpose of comparing the results of the use of two or more alternative economic policies. Second, spectral analysis is employed as a technique for validating an econometric model.

show abstract

Design of computer simulation experiments for industrial systems

Cited by 40 publications

References 38 publications

Tutorial: Simulation metamodeling

Tutorial: Simulation metamodeling

Methods for Factor Screening in Computer Simulation Experiments

Spectral Analysis of Data Generated by Simulation Experiments with Econometric Models

Contact Info

Product

Resources

About