Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems

Dai, Liyi

doi:10.1007/bf02190101

Cited by 144 publications

(86 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This algorithm was shown to converge in probability (and a.s. under certain added conditions [17]). Among other features, this approach is intended to exploit the fact that ordinal estimates are particularly robust with respect to estimation noise compared to cardinal estimates (see also [18]). The implication is that convergence of such algorithms is substantially faster.…”

Section: Introductionmentioning

confidence: 99%

Online Surrogate Problem Methodology for Stochastic Discrete Resource Allocation Problems

Gokbayrak

Cassandras

2001

Journal of Optimization Theory and Applications

View full text Add to dashboard Cite

We consider stochastic discrete optimization problems where the decision variables are non-negative integers. We propose and analyze an on-line control scheme which transforms the problem into a "surrogate" continuous optimization problem and proceeds to solve the latter using standard gradient-based approaches while simultaneously updating both actual and surrogate system states. It is shown that the solution of the original problem is recovered as an element of the discrete state neighborhood of the optimal surrogate state. For the special case of separable cost functions, we show that this methodology becomes particularly efficient. Finally, convergence of the proposed algorithm is established under standard technical conditions and numerical results are included in the paper to illustrate the fast convergence properties of this approach.

show abstract

Section: Introductionmentioning

confidence: 99%

Online Surrogate Problem Methodology for Stochastic Discrete Resource Allocation Problems

Gokbayrak

Cassandras

2001

Journal of Optimization Theory and Applications

View full text Add to dashboard Cite

show abstract

“…There are several ways to evaluate selection procedures, including the theoretical, empirical, and practical Dai (1996) proved exponential convergence for ordinal comparisons in certain conditions, so efficiency curves might be anticipated to be roughly linear on a semi-log scale,…”

Section: Evaluation Criteriamentioning

confidence: 99%

Selecting a Selection Procedure

2007

View full text Add to dashboard Cite

Selection procedures are used in a variety of applications to select the best of a finite set of alternatives. 'Best' is defined with respect to the largest mean, but the mean is inferred with statistical sampling, as in simulation optimization. There are a wide variety of procedures, which begs the question of which selection procedure to select. The main contribution of this paper is to identify, through extensive experimentation, the most effective selection procedures when samples are independent and normally distributed. We also (a) summarize the main structural approaches to deriving selection procedures, (b) formalize new sampling allocations and stopping rules, (c) identify strengths and weaknesses of the procedures, (d) identify some theoretical links between them, (e) and present an innovative empirical test bed with the most extensive numerical comparison of selection procedures to date. The most efficient and easiest to control procedures allocate samples with a Bayesian model for uncertainty about the means, and use new adaptive stopping rules proposed here.Selection procedures are intended to select the best of a finite set of alternatives, where best is determined with respect to the largest mean, but the mean must be inferred via statistical sampling (Bechhofer et al. 1995). Selection procedures can inform managers how to select the best of a small set of alternative actions whose effects are evaluated with simulation , and have been implemented in commercial simulation products. Selection procedures have also attracted interest in combination with tools like multiple attribute utility theory (Butler et al. 2001), evolutionary algorithms (Branke and Schmidt 2004), and discrete optimization via simulation (Boesel et al. 2003).Three main approaches to solving the selection problem are distinguished by their assumptions about how the evidence for correct selection is described and sampling allocations are made: the indifference zone (IZ, Kim and Nelson 2006), the expected value of information procedure (VIP, Chick and Inoue 2001a), and the optimal computing budget allocation (OCBA, Chen 1996) approaches. IZ procedures typically allocate samples in order to provide a guaranteed lower bound for the frequentist probability of correct selection (PCS), with respect to the sampling distribution, for selection problems in a specific class (e.g., the mean of the best is at least a prespecified amount better than each alternative). The VIP approach describes the evidence for correct selection with Bayesian posterior distributions, and allocates further samples using decision-theory tools to maximize the expected value of information in those samples. The OCBA is a heuristic that uses a normal distribution approximation for the Bayesian posterior distribution of the unknown mean performance of each alternative in order to sequentially allocate further samples. Each approach stipulates a number of different sampling assumptions, approximations, stopping rules and parameters that combine to define a pr...

show abstract

“…Our results buttress the literature on ordinal optimization see Ho et al (1992Ho et al ( , 2000, which focuses on the efficiency of ordinal comparisons rather than absolute estimation. In particular, the exponential convergence rate for static stochastic optimization problems is established in Dai (1996) and Dai and Chen (1997). Also, somewhat in the same spirit as our approach is the work of Robinson (1996) and Gürkan,Özge, and Robinson (1999), who consider sample path solution to stochastic variational inequalities, and establish conditions under which the sample path solution converges to the true solution; however, their setting is quite different from ours, in that we consider a dynamic model involving sequential decision making under uncertainty, and we focus on actually quantifying the error incurred in utilizing sample path estimates, going beyond just establishing convergence.…”

Section: Introductionmentioning

confidence: 99%

“…For finite horizon dynamic programming with finite space, backward induction can be used via Equation (3) to obtain the optimal value functions {J k (x), x ∈ S k , k = 1, 2, ...., T } and a corresponding optimal policy satisfying (4). For the infinite horizon case, value iteration, policy iteration, or variants on these are used to solve the stationary version of (3) when applicable.…”

Section: Introductionmentioning

confidence: 99%

“…When the transition probabilities are explicitly known, these procedures can sometimes be carried out in closed form or by using straightforward numerical procedures to calculate the necessary expectations. In our setting, based on sample paths of the MDP sequence X 1 , X 2 , ... for a given policy µ, the expectations in (1), (3), or (4), are estimated by taking sample means. By a sample path optimal policy, we mean a policy (possibly only partially specified, if not all states are visited in the sample paths) that optimizes the sample mean of the objective function given in (1).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence of Sample Path Optimal Policies for Stochastic Dynamic Programming

Fu¹,

Jin²

2005

View full text Add to dashboard Cite

ISR develops, applies and teaches advanced methodologies of design and analysis to AbstractWe consider the solution of stochastic dynamic programs using sample path estimates. Applying the theory of large deviations, we derive probability error bounds associated with the convergence of the estimated optimal policy to the true optimal policy, for finite horizon problems. These bounds decay at an exponential rate, in contrast with the usual canonical (inverse) square root rate associated with estimation of the value (cost-to-go) function itself.These results have practical implications for Monte Carlo simulation-based solution approaches to stochastic dynamic programming problems where it is impractical to extract the explicit transition probabilities of the underlying system model.

show abstract

Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems

Cited by 144 publications

References 11 publications

Online Surrogate Problem Methodology for Stochastic Discrete Resource Allocation Problems

Online Surrogate Problem Methodology for Stochastic Discrete Resource Allocation Problems

Selecting a Selection Procedure

Convergence of Sample Path Optimal Policies for Stochastic Dynamic Programming

Contact Info

Product

Resources

About