Recent advances in ranking and selection

Abstract

This tutorial provides an overview on recent advances made in ranking and selection (R&S) for selecting the best simulated system and discusses challenges that still exist in the field. We focus on indifference-zone R&S procedures that provide a guaranteed probability of correct selection when the best system is at least a user-specified amount better than the other systems.

A Flexible Evaluative Framework for Order Picking Systems

“…This approach requires an assumption about the functional form of utility functions and elicitation of the preference weights over multiple performance measures, and therefore difficulties can arise in implementation. Kim and Nelson (2007) state that the approach becomes inefficient for 20 or more systems. Evans et al (1991) point out that assessing multi-attribute value functions in multi-criteria optimization can face great difficulty in implementation when the number of criteria increases.…”

“…This methodology, however, still requires setting up an aspiration level for each constrained response beforehand, and the optimization is limited to one selected response. Other similar approaches include, for example, Andrado ´ttir et al ( 2005) and Batur and Kim (2005); see also Kim and Nelson (2007) for a discussion on the limitations of these methods.…”

“…This method also gives the minimal number of replications required to select the subset, while satisfying the user-defined probabilistic criterion. Recent advances in the R&S field provide more sophisticated models for specific issues, such as non-normality and dependence of simulation output data, and using common random numbers to increase experimental efficiency (Chick and Inoue 2001a, Kim and Nelson 2007, Tsai et al 2009). Sullivan and Wilson (1989) develop a similar approach, in which they use a more involved computational procedure to select subsets with a bounded random size; other approaches include simulation budget allocation (Chen et al 2000) and the Bayesian approach (Chick and Inoue 2001b).…”

Selection of the best with stochastic constraints

“…Although a lot of attention has been focused in the literature on Selection of the Best problems without stochastic constraints, the literature is surprisingly poor in proposing efficient methods for handling such constraints (Kim and Nelson 2007). Butler et.…”

Accounting for rank uncertainty in decision making for plant breeding

“…Ranking is an essential component of numerous decision-making processes where the selection of the best is of interest and there is therefore a great deal of related literature in different application domains. Statistical methods for subset selection and ranking-and-selection have been received extensive attention for decades (Gibbons et al, 1979;Seong-Hee Kim and Nelson, 2007), with many of those focus on determining how many samples are needed to make a selection (Dudewicz, 1980). In the type of plant breeding applications we address, however, the number of samples is fixed, and we need to address the uncertainty of the ranking considering this limitation.…”