Preferential Batch Bayesian Optimization

Abstract

Most research in Bayesian optimization (BO) has focused on direct feedback scenarios, where one has access to exact, or perturbed, values of some expensive-to-evaluate objective. This direction has been mainly driven by the use of BO in machine learning hyper-parameter configuration problems. However, in domains such as modelling human preferences, A/B tests or recommender systems, there is a need of methods that are able to replace direct feedback with preferential feedback, obtained via rankings or pairwise comparisons. In this work, we present Preferential Batch Bayesian Optimization (PBBO), a new framework that allows to find the optimum of a latent function of interest, given any type of parallel preferential feedback for a group of two or more points. We do so by using a Gaussian process model with a likelihood specially designed to enable parallel and efficient data collection mechanisms, which are key in modern machine learning. We show how the acquisitions developed under this framework generalize and augment previous approaches in Bayesian optimization, expanding the use of these techniques to a wider range of domains. An extensive simulation study shows the benefits of this approach, both with simulated functions and four real data sets.

The Impact of Expertise in the Loop for Exploring Machine Rationality

“…Therefore, it can propose examples using an acquisition function, ask the human to provide a choice, and then infer the underlying preference iteratively. When dealing with choices from pairwise comparisons, preferential Bayesian optimization (PBO) 1 as a specialized category of BO has received increasing development in recent studies [24,33,40,49,62]. While BO learns based on absolute rating utility (rate and assign a score to an option), PBO learns from human choice in pairwise comparisons according to Thurstone's law of comparative judgment [65].…”

“…While BO learns based on absolute rating utility (rate and assign a score to an option), PBO learns from human choice in pairwise comparisons according to Thurstone's law of comparative judgment [65]. To avoid the mentioned violations of the transitivity axiom, the recent extensions [7,40,62] to PBO transited from using a binary pairwise comparison to using a reasonable amount of options. These extensions can largely prevent violation of the transitivity axiom and infer more information at a time because they either consider choosing a set of options as winners among all given options [7,40]; or provide a ranking of all given options, where options may share the same level of rank [62].…”

Physically Consistent Preferential Bayesian Optimization for Food Arrangement

“…Gonzalez et al proposed a BO method for pairwise comparison and an improved acquisition function by focusing on the characteristics of the bandit problem [7]. Siivola et al proposed a PbBO method under batched query settings and discussed some approximate inferences [8].…”

“…By following [8], a Bayesian optimization (BO) [17] method finds w * = arg max w f (w) from as few queries as possible for black-box function f (w) : W → R, which is difficult to evaluate directly. Therefore, BO first constructs a surrogate model that is relatively easy to evaluate with a probabilistic model and then regresses f on it from set of queries {w i } K−1 i=0 and corresponding {f (w i )} K−1 i=0 .…”

“…evaluating them individually, supported by evidence from cognitive psychology [6]. Preference-based Bayesian Optimization (PbBO) [7], [8] is one of the methods to estimate preferences efficiently while generating such a two-choice query interactively. Hence, this paper considers the estimation problem of users' preferred food arrangement by PbBO with Computer Graphics (CG)-based dish images.…”