In this paper, we propose a stochastic search algorithm for solving general optimization problems with little structure. The algorithm iteratively finds high quality solutions by randomly sampling candidate solutions from a parameterized distribution model over the solution space. The basic idea is to convert the original (possibly non-differentiable) problem into a differentiable optimization problem on the parameter space of the parameterized sampling distribution, and then use a direct gradient search method to find improved sampling distributions. Thus, the algorithm combines the robustness feature of stochastic search from considering a population of candidate solutions with the relative fast convergence speed of classical gradient methods by exploiting local differentiable structures. We analyze the convergence and converge rate properties of the proposed algorithm, and carry out numerical study to illustrate its performance.
We study the empirical likelihood approach to construct confidence intervals for the optimal value and the optimality gap of a given solution, henceforth quantify the statistical uncertainty of sample average approximation, for optimization problems with expected value objectives and constraints where the underlying probability distributions are observed via limited data. This approach relies on two distributionally robust optimization problems posited over the uncertain distribution, with a divergence-based uncertainty set that is suitably calibrated to provide asymptotic statistical guarantees.
A large class of stochastic programs involve optimizing an expectation taken with respect to an underlying distribution that is unknown in practice. One popular approach to addressing the distributional uncertainty, known as the distributionally robust optimization (DRO), is to hedge against the worst case over an uncertainty set of candidate distributions. However, it has been observed that inappropriate construction of the uncertainty set can sometimes result in over-conservative solutions. To explore the middle ground between optimistically ignoring the distributional uncertainty and pessimistically fixating on the worst-case scenario, we propose a Bayesian risk optimization (BRO) framework for parametric underlying distributions, which is to optimize a risk functional applied to the posterior distribution of an unknown distribution parameter. Of our particular interest are four risk functionals: mean, mean-variance, value-at-risk, and conditional value-at-risk. To unravel the implication of BRO, we establish the consistency of objective functions and optimal solutions, as well as the asymptotic normality of objective functions and optimal values. More importantly, our analysis reveals a hidden interpretation: the objectives of BRO can be approximately viewed as a weighted sum of posterior mean objective and the (squared) half-width of the true objective's confidence interval.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.