A procedure to select the best subset among simulated systems using economic opportunity cost

Abstract

We consider subset selection problems in ranking and selection with tight computational budgets. We develop a new procedure that selects the best m out of k stochastic systems. Previous approaches have focused on individually separating out the top m from all the systems being considered. We reformulate the problem by casting all m-sized subsets of the k systems as the alternatives of the selection problem. This reformulation enables our derivation to follow along traditional ranking and selection frameworks. In particular, we extend the value of information procedure to subset selection. Furthermore, unlike previous subset selection efforts, we use an expected opportunity cost (EOC) loss function as evidence for correct selection. In minimizing the EOC, we consider both deriving an asymptotic allocation rule as well as approximately solving the underlying optimization problem. Experiments show the advantage of our approach for tests with small computational budgets.

Energy-Efficient Urban Traffic Management: A Microscopic Simulation-Based Approach

“…We are addressing SO signal control problems that account for greenhouse gas emissions and various pollutant emissions (Osorio and Nanduri 2015). Additionally, to better address problems with high-variance performance measures, we are developing simulation-budget allocation strategies that determine the number of simulation replications (i.e., sample size) to allocate across various points (Chingcuanco and Osorio 2013). The technique, known as a subset selection procedure, performs well for small sample size problems, such as those considered in this paper.…”

A Practical Approach to Subset Selection for Multi-objective Optimization via Simulation

“…Numerical results suggest that OCBASS is more efficient than OCBA-m, with improvements in efficiency of around 29.2 and 42.5%. Chingcuanco and Osorio [2013] consider the same problem but have a different approach in which they use all of the possible subsets of size m as the different designs in a ranking and selection problem. Gao and Chen [2015] instead minimize the Expected Opportunity Cost (EOC), measured as the difference in expected performance between the selected designs and the best designs.…”

On the optimal computing budget allocation problem for large scale simulation optimization

“…Meanwhile, the E ( OC ) has become important in many applications in business and engineering that led to a recently new selection procedure in order to reduce the opportunity cost of a potentially incorrect selection. For more details, see Gupta and Miescke [35,36] Chick and Inoue [37,38] , Chick and Wu [39] , Chingcuanco and Osorio [40] , and Gao and Chen [41] .…”