Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Times

Abstract

Computing a Gaussian process (GP) posterior has a computational cost cubical in the number of historical points. A reformulation of the same GP posterior highlights that this complexity mainly depends on how many unique historical points are considered. This can have important implication in active learning settings, where the set of historical points is constructed sequentially by the learner. We show that sequential black-box optimization based on GPs (GP-Opt) can be made efficient by sticking to a candidate solution for multiple evaluation steps and switch only when necessary. Limiting the number of switches also limits the number of unique points in the history of the GP. Thus, the efficient GP reformulation can be used to exactly and cheaply compute the posteriors required to run the GP-Opt algorithms. This approach is especially useful in real-world applications of GP-Opt with high switch costs (e.g. switching chemicals in wet labs, data/model loading in hyperparameter optimization). As examples of this meta-approach, we modify two well-established GP-Opt algorithms, GP-UCB and GP-EI, to switch candidates as infrequently as possible adapting rules from batched GP-Opt. These versions preserve all the theoretical no-regret guarantees while improving practical aspects of the algorithms such as runtime, memory complexity, and the ability of batching candidates and evaluating them in parallel.

(Private) Kernelized Bandits with Distributed Biased Feedback

“…Putting the two stages together, we have the above result. Thus, it remains to bound the number of batches H l within each phase l. Fortunately, inspired by [Cal22], we are able to show that H l can be upper bounded by the maximum information gain. We state this result in Lemma 5.6 and provide the proof in Appendix D.…”

“…As we already know, GP-UCB has a computation complexity of O(|D|T 3 ), because it requires computing the posterior mean and variance using O(T 2 ) and then finds the action that maximizes the UCB function per step. Recently, BBKB in [Cal20] improves the time complexity to (|D|T γ 2 T ), and later MINI-GP-Opt in [Cal22] further reduces computation complexity to O(T + |D|γ 3 T + γ 4 T ), which is currently the fastest no-regret algorithm. Although more feedback is needed to address the additional bias in our setting, our algorithm can still achieve an improvement with the highest order term being O(γ T T α ).…”

“…Lemma A.2 (Proposition A.1 in [Cal22]/Lemma 4 in [Cal20]). For any kernel k, set of points X τ , x ∈ D, and τ < τ , we have…”

“…Then, play this action for T l (a h ) (C 2 − 1)/Σ 2 h−1 (a h ) rounds, which forms the h-th batch. Here, the batch size schedule is inspired by the rare-switching idea in [APS11;Cal22]. This batch schedule strategy enables us to merge rounds and thus shrink the dimensions of the matrix and vectors used for computing the variance in Eq.…”

“…To solve this new problem, a learning algorithm needs to be able to learn the unknown function from such biased feedback in a sample-efficient manner. Moreover, two practical challenges naturally arise in our problem: communication cost due to distributed learning [Che21a] and computation complexity due to GP update [Cal22]. Therefore, not only need the learning algorithms be sample-efficient, but they must also be scalable in terms of both communication efficiency and computation complexity.…”