Bayesian Optimization in a Billion Dimensions via Random Embeddings

High-dimensional multi-fidelity Bayesian optimization for quantum control

“…Many methods have been proposed to address the problems of BO in the high dimensional spaces. Most of them use the idea of optimizing the objective function in a low dimensional space by mapping the high dimensional space to a low dimensional space [52][53][54][55][56].…”

Gradient and uncertainty enhanced sequential sampling for global fit

“…Therefore, global sensitivity analysis [58] could be used to select only input variables which have important contribution to the response variability. Alternatively, embeddings based on linear [59,60] or non-linear projections [61,62] could be used to learn a reduced latent representation of the input. However, none of these methods are efficient if the response function is sensitive to all dimensions.…”

Active Learning of Linear Embeddings for Gaussian Processes

“…Hence data is effectively ignored during most predictions, and learning is impossible. Practical experience shows, however, that many functions are insensitive to some of their inputs (Wang et al, 2013), thus have low effective dimensionality (Figure 1). Our goal is to discover a R ∈ R d×D such that, for low-dimensional…”

“…Within Bayesian optimization, much recent work has focused on high-dimensional problems (Hutter et al, 2011;Chen et al, 2012;Carpentier & Munos, 2012;Bergstra & Bengio, 2012;Hutter, 2009). Recently, Wang et al (2013) proposed using randomly generated linear embeddings. In contrast, our active learning strategy can provide an initialization phase that selects objective function evaluations so as to best learn low-dimensional structure.…”