Identifying efficient solutions via simulation: myopic multi-objective budget allocation for the bi-objective case

Abstract

Simulation optimisation offers great opportunities in the design and optimisation of complex systems. In the presence of multiple objectives, there is usually no single solution that performs best on all objectives. Instead, there are several Pareto-optimal (efficient) solutions with different trade-offs which cannot be improved in any objective without sacrificing performance in another objective. For the case where alternatives are evaluated on multiple stochastic criteria, and the performance of an alternative can only be estimated via simulation, we consider the problem of efficiently identifying the Pareto-optimal designs out of a (small) given set of alternatives. We present a simple myopic budget allocation algorithm for multi-objective problems and propose several variants for different settings. In particular, this myopic method only allocates one simulation sample to one alternative in each iteration. This paper shows how the algorithm works in bi-objective problems under different settings. Empirical tests show that our algorithm can significantly reduce the necessary simulation budget.

Performance Metrics for Multi-Objective Optimisation Algorithms Under Noise

“…[8] additionally propose to measure the "noise misinformation", which they define as the average distance between the returned solutions' predicted and true fitness values. In the context of multi-objective ranking and selection, [14] proposed to optimise a metric called hypervolume difference, motivated by the challenge for a DM to pick the correct solution in the end. The orange solution is erroneously included as Pareto optimal even though it is not.…”

Multi-Fidelity Simulation Modeling for Discrete Event Simulation: An Optimization Perspective

“…2) The DES optimization problem is an expensive, noisy, and black-box problem [23], [24]. We follow the idea of EA-based multi-objective simulation optimization methods and develop a new method called MOSO-HV that inherits the merits of the S Metric Selection Evolutionary Multi-objective Optimization Algorithm (SMS-EMOA) [25] and the Myopic Multi-objective Budget Allocation based on Hypervolume (M-MOBA-HV) [26].…”

“…To alleviate the adverse effects of noisy observations on the evaluation of the hypervolume, a myopic simulation budget allocation rule M-MOBA-HV is used in the evaluation step. Distinguished from existing methods, M-MOBA-HV is myopic and only allocates a few simulation budgets to one solution without asymptotic approximations [26]. M-MOBA-HV only requires small samples, matching the iterative optimization algorithm SMS-EMOA.…”

“…Any change in the estimation leads to the difference in hypervolume. M-MOBA-HV utilizes the hypervolume difference to quantify the change of the perceived Pareto front before and after allocating simulation budgets to the candidate solution [26]. Given a solution set and performance observations, we assign the incremental simulation budgets to the point x with the maximum expected hypervolume difference.…”

Ranking and selection for pairwise comparison

“…For example, several studies proposed efficient computing budget allocation rules for selecting the optimal subset (Chen et al, 2008; Gao & Chen, 2016; Zhang et al, 2015). In the case of designs with multiple performance measures, Lee et al (2010) and Branke and Zhang (2019) developed efficient simulation allocation procedures for selecting the nondominated Pareto set. For stochastic constrained simulation optimization problems, simulation budget allocation rules have been developed in Lee et al (2012), Hunter and Pasupathy (2013), and Xiao et al (2019).…”