New Procedures to Select the Best Simulated System Using Common Random Numbers

Chick, Stephen E.; Inoue, Koichiro

doi:10.1287/mnsc.47.8.1133.10229

Cited by 113 publications

(76 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We recorded the iteration number k when there was a change in N k . For example, N k remained at 3 in iterations [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19], and N k changed to 4 at iteration 20. Since in the first 19 iterations, the averaged sample function wasf 3 , all the steps were taken regardinĝ f 3 as the objective function.…”

Section: Numerical Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Variable-Number Sample-Path Optimization

Deng

Ferris

2007

Math. Program.

View full text Add to dashboard Cite

The sample-path method is one of the most important tools in simulation-based optimization. The basic idea of the method is to approximate the expected simulation output by the average of sample observations with a common random number sequence. In this paper, we describe a new variant of Powell's UOBYQA (Unconstrained Optimization BY Quadratic Approximation) method, which integrates a Bayesian Variable-Number Sample-Path (VNSP) scheme to choose appropriate number of samples at each iteration. The statistically accurate scheme determines the number of simulation runs, and guarantees the global convergence of the algorithm. The VNSP scheme saves a significant amount of simulation operations compared to general purpose 'fixed-number' sample-path methods. We present numerical results based on the new algorithm.Keywords sample-path method, simulation-based optimization, Bayesian analysis, trust region method Dedication This paper is dedicated to Stephen Robinson on the occasion of his 65th birthday. The authors are grateful for his encouragement and guidance over the past two decades, and the inspirational work he has done in the topic of this paper.

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

“…There is an extensive literature on using Bayesian methods in simulation output analysis. For example, Chick and Inoue [3,4] has implemented Bayesian estimation in ordering discrete simulation systems (ranking and selection [1,18]). Deng and Ferris [8] propose a similar Bayesian analysis to evaluate the stability of surrogate models.…”

Section: Introductionmentioning

confidence: 99%

Variable-Number Sample-Path Optimization

Deng

Ferris

2007

Math. Program.

View full text Add to dashboard Cite

show abstract

“…There are two basic approaches: (i) how to select, with high probability, the system, decision, or policy that is-for practical purposes-the best of the potential choices; and (ii) how to screen the potential systems, decisions, or policies to obtain a (random-size) subset of "good" ones. Many procedures have been developed specifically to address some of the characteristics of simulation experiments we discuss in §3 (Chick and Inoue 2001, Hsu 1996, Goldsman et al 2002, Nelson and Goldsman 2001. Some assume that all populations are compared with each other, whereas others assume comparisons with a standard.…”

Section: Finding Robust Decisions or Policiesmentioning

confidence: 99%

State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments

Kleijnen

Sanchez

Lucas

et al. 2005

INFORMS Journal on Computing

382

212

View full text Add to dashboard Cite

Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models. We discuss a toolkit of designs for simulators with limited DOE expertise who want to select a design and an appropriate analysis for their experiments. Furthermore, we provide a research agenda listing problems in the design of simulation experiments-as opposed to real-world experiments-that require more investigation. We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system, (2) finding robust decisions or policies as opposed to so-called optimal solutions, and (3) comparing the merits of various decisions or policies. Our discussion emphasizes aspects that are typical for simulation, such as having many more factors than in real-world experiments, and the sequential nature of the data collection. Because the same problem type may be addressed through different design types, we discuss quality attributes of designs, such as the ease of design construction, the flexibility for analysis, and efficiency considerations. Moreover, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, complicated metamodels require more simulation runs. We present several procedures to validate the metamodel estimated from a specific design, and we summarize a case study illustrating several of our major themes. We conclude with a discussion of areas that merit more work to achieve the potential benefits-either via new research or incorporation into standard simulation or statistical packages.

show abstract

“…We apply a class of optimal learning methods known variously as value of information procedures (see Chick and Inoue 2001a, Chick and Inoue 2001b, or Chick 2006 and knowledge gradient policies (Frazier et al 2008). Our Bayesian belief about player skill induces a probability distribution on the outcome of the next game, and thus, on the future beliefs that we will use to make future decisions.…”

Section: Introductionmentioning

confidence: 99%

May the best man win: Simulation optimization for match-making in e-sports

Ryzhov¹,

Tariq²,

Powell³

2011

Proceedings of the 2011 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

We consider the problem of automated match-making in a competitive online gaming service. Large numbers of players log on to the service and indicate their availability. The system must then find an opponent for each player, with the objective of creating competitive, challenging games that do not heavily favour either side, for as many players as possible. Existing mathematical models for this problem assume that each player has a skill level that is unknown to the game master. As more games are played, the game master's belief about player skills evolves according to a Bayesian learning model, allowing the game master to adaptively improve the quality of future games as information is being collected. We propose a new decision-making policy in this setting, based on the knowledge gradient concept from the literature on optimal learning. We conduct simulations to demonstrate the potential of this policy.

show abstract

New Procedures to Select the Best Simulated System Using Common Random Numbers

Cited by 113 publications

References 18 publications

Variable-Number Sample-Path Optimization

Variable-Number Sample-Path Optimization

State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments

May the best man win: Simulation optimization for match-making in e-sports

Contact Info

Product

Resources

About