Sampling Acquisition Functions for Batch Bayesian Optimization

Alessandro De Palma Celestine Mendler-DunnerUniversity of Oxford UC Berkeley

Thomas Parnell, Andreea Anghel, Haralampos PozidisIBM Research, Zurich

Abstract

We present Acquisition Thompson Sampling(ATS), a novel technique for batch BayesianOptimization (BO) based on the idea of sam-pling multiple acquisition functions from astochastic process. We define this processthrough the dependency of the acquisitionfunctions on a set of model hyper-parameters.ATS is conceptually simple, straightforwardto implement and, unlike other batch BOmethods, it can be employed to parallelizeany sequential acquisition function or tomake existing parallel methods scale fur-ther. We present experiments on a varietyof benchmark functions and on the hyper-parameter optimization of a popular gradi-ent boosting tree algorithm. These demon-strate the advantages of ATS with respect toclassical parallel Thompson Sampling for BO,its competitiveness with two state-of-the-artbatch BO methods, and its effectiveness ifapplied to existing parallel BO algorithms.

1 Introduction

Bayesian optimization (BO) methods deal with the op-timization of expensive, black-box functions, i.e., func-tions with no closed-form expression or derivative in-formation. Many such problems arise in practice infields such as: hyper-parameter optimization of ma-chine learning algorithms (Snoek et al., 2012), synthe-sis of short polymer fiber materials (Li et al., 2017),and analog circuit design (Lyu et al., 2018). There-fore, in recent years, BO gained widespread popular-ity to the point that its versatility was showcased ontoy problems like optimizing chocolate cookie recipes(Solnik et al., 2017).

From a high-level perspective, BO works by traininga probabilistic regression model (typically, GaussianProcesses) on the available function evaluations. Thismodel is then employed to decide where to evaluatethe function next, maximizing a trade-off between ex-ploration and exploitation that is formalized throughan acquisition function a(x). Many sequential BO al-gorithms exist, differing by the choice of the regres-sion model, the acquisition function, or properties ofthe input domain (Hutter et al., 2011; Srinivas et al.,2012; Shahriari et al., 2016a; Springenberg et al., 2016;Hernandez-Lobato et al., 2015). Recently, parallel BOtechniques have gained popularity. These methods se-lect multiple points to be evaluated in parallel in orderto better exploit modern hardware capabilities, and tospeed up the overall optimization process in the caseof costly evaluations like training machine learning al-gorithms. In the parallel case, typically, the acquisi-tion function must return q points at once (batch BO),these being greedily selected one after the other.

Most successful batch BO algorithms focus on par-allelizing a specific acquisition function (Kandasamyet al., 2018; Wu and Frazier, 2016; Daxberger and Low,2017; Shah and Ghahramani, 2015; Gupta et al., 2018).In this paper, instead, we will focus on methods thatcan be applied to any sequential acquisition function,as in general no single one consistently outperforms theothers (Hoffman et al., 2011). Amongst these generalmethods, the prevalent approach is to modify the land-scape of the acquisition function a(x) after a point isscheduled so as to avoid returning another point in itsneighborhood. For instance, Local Penalization (LP)(Gonzalez et al., 2016) accomplishes this by suppress-ing (penalizing) the value of a(x) in the neighborhoodof any scheduled x using an approximation of the Lip-schitz constant of the unknown function. A family ofalgorithms including, e.g., B-LCB (Desautels et al.,2014), instead, obtains the same effect by hallucinat-ing the results of the points that have been scheduledbut not yet completed.

arX

iv:1

903.

0943

4v2

[cs

.LG

] 1

6 O

ct 2

019

Sampling Acquisition Functions for Batch Bayesian Optimization

1.1 Intuition, Contributions, Challenges

In general, a sequential acquisition function dependson a vector of parameters mostly belonging to the em-ployed regression model. We will denote the parame-ter vector by θ, and write a(x) = a(x,θ) to refer toa member of the family of acquisition functions. Theθ vector exerts a strong effect on the practical perfor-mance of BO (Snoek et al., 2012). Hence, in practice,it is either set to its maximum likelihood value, or itsinfluence is mitigated by using an acquisition functionaveraged over a fixed number of possible values of θ.Our intuition is that, rather than focusing on modi-fying the landscape of a fixed a(x, {θi}), where {θi}possibly denotes a set of vectors, such a dependencycan be exploited to select each point in a batch froma different member of the family. In order to do so,inspired by Parallel Thompson Sampling (P-TS) (Kan-dasamy et al., 2018), we interpret a given sequentialacquisition function a(x, {θi}) as being sampled froma stochastic process. It then suffices to draw samplesfrom the process to obtain as many different acquisi-tion functions as the number of batch points to evalu-ate. Our contributions are the following:

• We present ATS, a novel batch Bayesian optimiza-tion algorithm based on the idea of sampling ac-quisition functions from a stochastic process definedby their dependency on hyper-parameters θ. ATSis conceptually simple and easy to implement. Incontrast to the previously published randomized ap-proach (Kandasamy et al., 2018), ATS can paral-lelize any arbitrary sequential acquisition functionand can be employed to make existing parallel ac-quisition functions scale further.

• We show how existing techniques to diversify pointswithin a batch, can be incorporated into our ran-domized framework to improve performance of ATSon highly multi-modal functions.

• We present results on five well-known benchmarkfunctions and on hyper-parameter tuning for XG-Boost (Chen and Guestrin, 2016). When applied tosequential acquisition functions, we demonstrate theadvantages of ATS compared to P-TS (Kandasamyet al., 2018), B-LCB (Desautels et al., 2014) andLP (Gonzalez et al., 2016), three popular state-of-the-art batch BO methods. In particular, ATS con-sistently finds better minima than P-TS.

• We show that ATS can be successfully used to im-prove the performance of existing parallel methods,P-TS and B-LCB, on our benchmark functions.

We leave the formulation of regret bounds as futurework. Most regret bounds (sequential and parallel),such as those for LCB (Srinivas et al., 2012), B-LCB (Desautels et al., 2014), TS (Chowdhury and

Gopalan, 2017), P-TS (Kandasamy et al., 2018) allowonly fixed hyper-parameters (Shahriari et al., 2016b).In practice, hyper-parameters are changed dynami-cally in the optimization (set to a point estimate)or approximately marginalized out. The former ap-proach might not converge to the global minimum(Bull, 2011) unless, as it was very recently shown, fur-ther constraints are imposed on the hyper-parameters(Berkenkamp et al., 2019). We employ marginaliza-tion: to the best of our knowledge, a regret bound ismissing even under a fully Bayesian treatment for se-quential BO (Shahriari et al., 2016b). In addition, wewould need to prove that hyper-parameter diversityspeeds up parallel convergence. Therefore, we believethat regret bounds for ATS would be an interestingand challenging direction for future work also in thepurely sequential context.

2 Problem Statement and Background

Our goal is to find the minimum of a black-box func-tion f : X → R, defined on a bounded set X ⊂ Rd.While the function itself is unknown, we have access tothe results of its evaluations f(x) at any domain pointx. We are interested in scenarios where these evalua-tions are costly and we seek to minimize f using as fewevaluations as possible. For instance, f might repre-sent the loss incurred by a machine learning algorithmtrained with the hyper-parameter vector x.

2.1 Bayesian Optimization

Bayesian Optimization (BO) approaches this black-box optimization problem by placing a prior on f andupdating the model by conditioning on the evaluationsas they are collected. Once the surrogate model is up-dated, BO employs it to select the next point to eval-uate according to a criterion referred to as the acqui-sition function. This function should not only exploitthe regions that are likely to contain the minimum,but also explore those where uncertainty is significant.

Gaussian Processes (GP) are a common and ver-satile choice for the function prior. Under a GP,any finite number of function outputs is jointly dis-tributed according to a multivariate Gaussian distri-bution N (µ,K), where the covariance K of the Gaus-sian process is defined through a kernel function k,parametrized by θk as Ki,j = k(xi,xj |θk). The choiceof the kernel function k determines the family of func-tions that a GP can represent. We use the Matern 5/2kernel due to its widespread usage and its good em-pirical performance on tuning machine learning algo-rithms (Snoek et al., 2012). Moreover, we assume theprior mean µp to be constant (yet possibly non-zero)on the input domain. In general, while the function

A. De Palma, C. Mendler-Dunner, T. Parnell, A. Anghel, H. Pozidis

f is deterministic, its evaluations may be noisy. It is,therefore, common practice to assume Gaussian noisein addition to the GP, so that yi = f(xi) +N (0, σ2

nI).We will denote the set of collected observations up toevaluation t with Dt = {(xi, yi)) | i < t}. A desirableproperty of GPs in this context is that N (µ,K|Dt) isagain a GP, and its posterior distribution at a pointcan be computed in closed form in O(t3) time (Ras-mussen and Williams, 2006). In the remainder of thepaper, we will ignore the evaluation noise and setyi = f(xi) to facilitate the presentation. In orderto generalize to the noisy case, it suffices to substi-tute f(xi) with yi and to include the noise in thekernel matrix. Once t evaluations are available, thenext point is selected by maximizing the acquisitionfunction: xt+1 = arg maxX a(x|Dt), where the condi-tioning explicitly denotes the dependency on collectedevaluations Dt through the updated surrogate model.Two widespread sequential acquisition functions areExpected Improvement (EI) (Jones et al., 1998) andLower Confidence Bound (LCB) (Srinivas et al., 2012):

EI(x|Dt) = Ef(x|Dt)[max(f(x−)− f(x), 0)]

LCB(x|Dt) = σ(x|Dt)− µ(x|Dt)

where f(x−) denotes the best (smallest) evaluationresult recorded until evaluation t.

2.2 Parallel Bayesian Optimization

Bayesian Optimization as presented in the previoussection is an inherently sequential algorithm: pointsare evaluated one by one, and the model is up-dated to reflect the evaluation results. Many workshave explored the possibility of evaluating more thanone point at once, either asynchronously (Snoeket al., 2012; Kandasamy et al., 2018) or synchronously(Gonzalez et al., 2016; Desautels et al., 2014; Guptaet al., 2018; Kathuria et al., 2016; Lyu et al., 2018;Daxberger and Low, 2017). Next, we will focus onthe latter, more studied, case, as asynchronous algo-rithms can also be employed in a synchronous fashion.Some approaches, like q-EI by Wang et al. (2016), the-oretically generalize sequential acquisition functions tothe case in which q points must be returned at once.Unfortunately, such approaches are not computation-ally feasible for q > 4, as the batch of points needsto be jointly optimized (Daxberger and Low, 2017).Therefore, most of the attention has been devoted togreedy methods, where points are selected one afterthe other. ATS falls within this category. We will callpending evaluations, the points for which an evaluationhas been scheduled but not yet completed. We willdenote all pending evaluations with the prime sign: ifevaluation i is pending, we will write x′i, meaning thatx′i is available, but not f(x′i).

A plethora of approaches for greedy batch BO exists(Gonzalez et al., 2016; Desautels et al., 2014; Guptaet al., 2018; Kathuria et al., 2016; Lyu et al., 2018;Daxberger and Low, 2017; Snoek et al., 2012; Kan-dasamy et al., 2018); here, we will describe those thatform the background to our algorithms. A well-knownheuristic is to hallucinate the results of the pendingevaluations, i.e., to approximate them with knownquantities. This way, the acquisition function is up-dated from a(x|Dt) to a(x|Dt∪{(x′i, hi) ∀ x′i}), wherehi denotes the hallucinated evaluation result for thepending evaluation x′i. For instance, the very effectivebatch LCB (B-LCB) approach updates the surrogatemodel after each batch point has been scheduled bysetting f(x′i) = µ(x∗|Dt) (Desautels et al., 2014). Thebatch BO algorithm by Gupta et al. (2018), instead,parametrizes the LCB acquisition function to expressdifferent explore-exploit trade-offs and, arguing thatdifferent trade-offs are associated to different maximaof a(x|Dt), assigns each batch point a new trade-off.

2.3 Thompson Sampling

Thompson Sampling (TS) (Thompson, 1933) is arather old heuristic to address the explore-exploitdilemma that gained popularity in the context ofmulti-armed bandits (Chapelle and Li, 2011). Thegeneral idea is to choose an action according to theprobability that it is optimal. Applied to BO, this cor-responds to sampling a function g from the GP pos-terior and selecting xt+1 = arg maxX g (Kandasamyet al., 2018). In fact, let p∗(x) be the probability thatpoint (action) x optimizes the GP posterior, then:

p∗(x) =

∫p∗(x|g) p(g|Dt) dg

=

∫δ(x− arg min

Xg(x))p(g|Dt) dg. (1)

where δ denotes the Dirac delta distribution. TS istherefore equivalent to a randomized sequential acqui-sition function, one that is easy to parallelize. In fact,it suffices to sample multiple functions from the GPposterior to get a batch of points to be evaluated inparallel (P-TS) (Kandasamy et al., 2018). In this pa-per, we generalize the reasoning in (1) so as to paral-lelize an arbitrary acquisition function, and addition-ally incorporate ideas from hallucinations and intra-batch trade-off differentiation.

3 Sampling Acquisition Functions

We first introduce Acquisition Thompson Sampling(ATS), our novel algorithm for both batch and asyn-chronous parallel Bayesian Optimization (§3.1) andthen discuss two extensions thereof, j-ATS and h-ATS

Sampling Acquisition Functions for Batch Bayesian Optimization

(§3.2). The common idea behind these methods is tosample an acquisition function from a stochastic pro-cess for each parallel evaluation to perform.

3.1 Acquisition Thompson Sampling (ATS)

Any acquisition function a(x) explicitly depends onthe surrogate model’s hyper-parameters θ and can betherefore written as a(x,θ).

Sequential treatment of θ. The most commonway to choose θ is to optimize the marginal likelihoodunder the Gaussian process. As acquisition functionsare very sensitive to these hyper-parameters, Snoeket al. (2012) suggest the adoption of a fully Bayesianapproach by marginalizing over them:

a(x|Dt) =

∫a(x,θ|Dt) p(θ|Dt) dθ (2)

As the exact marginalization in (2) is intractable, itis approximated by as(x), the empirical average ofs samples from the data posterior of θ obtained viaMarkov Chain Monte Carlo (MCMC) sampling (Snoeket al., 2012):

as(x|Dt) =1

s

s∑q=1

a(x,θq|Dt) s.t. θq ∼ p(θ|Dt) (3)

Exploiting θ for parallelism. In this paper, weaim to leverage the sensitivity to θ for selecting a di-verse batch of points to be evaluated in parallel. Inorder to do so, we propose an alternative interpreta-tion of (3). The main intuition is to recognize that,for low s, as(x) represents a sample from a probabilitydistribution over functions that is implicitly defined bythe θ data posterior, i.e., a stochastic process. We willdenote such a process by as(x|Dt), and write as ∼p(as|Dt) to refer to the sampling in (3). We can thenargue that the sequential criterion for determining thenext point to evaluate, xt+1 = arg maxX as(x|Dt), canbe seen as sampling from pa∗

s(x), the probability that

x maximizes the stochastic process. We can write:

pa∗s(x) =

∫pa∗

s(x|as) p(as|Dt) das (4)

=

∫δ(x− arg max

Xas(x|Dt))p(as|Dt) das

In light of this interpretation, a sequential acquisi-tion function can be employed in the parallel settingby sampling as many different acquisition functionsfrom as(x|Dt) (i.e., as many approximations of themarginalized function a(x|Dt)) as the parallel evalua-tions to perform. We call this procedure ATS as ouralgorithm closely resembles Thompson Sampling for

Algorithm 1 Batch iteration, Acquisition ThompsonSampling (ATS).

input dataset Dt, acquisition function to parallelizea(x,θ|Dt), batch size M > 0, s > 0.

1: for i = 1, . . . ,M in parallel do2: sample s new GP hyper-parameter vectors ac-

cording to θq ∼ p(θ|Dt).

3: as(x|Dt)← 1s

∑sq=1 a(x,θq|Dt).

4: xt+i ← arg maxX as(x|Dt)5: evaluate f(xt+i).

6: Dt+M ← Dt ∪ {(xk, f(xk)) ∀ t < k ≤ t+M} .

Bayesian Optimization (cf. §2.3). In fact, (4) can beinterpreted as an adaptation of (1) to a case in whichthe optimal action is defined with respect to a stochas-tic acquisition function rather than the GP posterior.While P-TS is an acquisition function of its own, thisadaptation allows us to obtain a general approach, ap-plicable for the parallelization of any sequential acqui-sition function by means of its parametrization θ. InSection 4, we demonstrate empirically that the as sam-ples are diverse enough to provide significant speed-ups to sequential BO, and that ATS consistently findsbetter minima than P-TS. Algorithm 1 outlines thepseudo-code for a single iteration of the batch versionof ATS. For implementation details, including the cho-sen prior for θ and the employed MCMC sampling al-gorithm, cf. Appendix B.

The role of s. The parameter s plays a fundamentalrole in ATS, as it controls the trade-off between sam-ple diversity and surrogate model reliability. In fact,s → ∞ yields perfect marginalization and no paral-lelism, while s = 1 implies significant batch diversitybut a poor representation of the black-box function f .Therefore, s should be chosen to be a monotonicallydecreasing function of the batch size M . In practice,we found s = 10 to be a good compromise betweensample diversity and surrogate model reliability for ourtarget functions and levels of parallelism (cf. § 4).

Enhancing parallel acquisition functions. Dueto its generality, ATS can be also employed on top ofan existing greedy parallel approach to make it scalefurther. In fact, a(x,θ|Dt) in Algorithm 1 could rep-resent batch BO acquisition functions like P-TS or B-LCB. For the latter, in addition to hallucinating thesurrogate model, one would sample new hallucinatedfunctions from the stochastic process for every batchpoint, with the goal of increasing batch diversity. Inorder to match the properties of the chosen parallel ac-quisition function with a variable degree of accuracy,we toss a biased coin for each point, sampling a newfunction from the process with probability p.

A. De Palma, C. Mendler-Dunner, T. Parnell, A. Anghel, H. Pozidis

3.2 Techniques for Batch Diversity

If f exhibits multiple minima, ATS might over-exploitthe current posterior of the function, producing abatch of points centered around a local minimum. Wepropose two variants of ATS to address this issue.

Explore-exploit trade-offs. Pursuing different ex-plore-exploit trade-offs within a batch encourages theexploration of regions of the input domain with vary-ing degrees of posterior uncertainty. This idea wasimplemented in Gupta et al. (2018) by explicitlysolving a multi-objective optimization problem onLCBj(x|Dt) = jσ(x|Dt) − µ(x|Dt), a parametrizedversion of LCB, and taking M different trade-offs fromthe resulting Pareto curve (one per batch point). Inthis context, the parameter j is commonly called jitter:the larger the jitter, the more explorative the acquisi-tion function. We build on the intuition of Gupta et al.(2018), but rather than explicitly solving the multi-objective optimization, we translate this idea into ourrandomized framework by sampling different trade-offsfrom a distribution. In order to do this, we associatea new process as,j(x|Dt) to the parametrization, andplace an independent prior on j so as to sample jitteralongside the θi vectors. In addition to LCB, such aparametrization is commonly employed for many pop-ular acquisition functions; e.g., EI (the GPyOpt au-thors, 2016):

EIj(x|Dt) = Ef(x|Dt)[max(f(x−)− j − f(x), 0)]

As a consequence, sampling from as,j(x|Dt) im-plies obtaining acquisition functions having differentexplore-exploit trade-offs. Pseudo-code for a batch it-eration of this jittered variant (j-ATS) of the algo-rithm, along with details of the employed data poste-rior for j, can be found in Appendix B.

Hallucinating Pending Evaluations. The maindrawback of j-ATS lies in the loss of generality thatarises from the need of using an acquisition function-specific prior. As an alternative, we propose to addhallucinations of pending evaluations (see § 2.2) to thedata posterior of θ. This is in contrast with previouswork, where hallucinations are added to the surrogatemodel (Desautels et al., 2014). The resulting method,h-ATS, is fully described in Appendix A.2.

4 Experimental Results

This section presents the results of our empirical eval-uations for the proposed ATS algorithm (and its vari-ants), applied both to sequential and parallel acquisi-tion functions. In Section 4.1 we benchmark on well-known synthetic functions from the global optimiza-tion literature and in Section 4.2 we present the results

on the hyper-parameter tuning of a gradient boostingtree algorithm on a binary classification task.

Methodology & Baselines. We compare with twostate-of-the-art, well-known batch Bayesian optimiza-tion algorithms: Local Penalization (LP) (Gonzalezet al., 2016) and batch LCB (B-LCB) (Desautels et al.,2014), and against Parallel ”classical” Thompson Sam-pling (P-TS) (Kandasamy et al., 2018). LP was takenfrom the implementation available within the open-source GPyOpt package (the GPyOpt authors, 2016).We instead re-implemented B-LCB and P-TS withinour framework. While LP is a general batch BO ap-proach (applicable to any acquisition function), B-LCB was presented and analyzed only for LCB 1 andP-TS parallelizes the TS acquisition function only.Both B-LCB and LP are very competitive againstmany of the more complex, non-general algorithmspresented in the last two years (Daxberger and Low,2017; Gupta et al., 2018; Lyu et al., 2018; Kathuriaet al., 2016). Moreover, their performance is superiorto other algorithms commonly employed as baselines,such as q-KG (Wu and Frazier, 2016) and q-EI (Cheva-lier and Ginsbourger, 2013); we hence forgo such acomparison.

We employ two widely used acquisition functions, LCBand EI (cf. Section 2.1) for LP, and implement ATSfor EI, LP, B-LCB and P-TS. For ATS applied on topof parallel acquisition functions, we sample a new a(x)for each point with probability p = 1 or p = 0.5. Forall algorithms, we employ a GP surrogate model withMatern 5/2 kernel (Snoek et al., 2012) and (exceptfor LP, whose original implementation does not sup-port it) we approximately marginalize the acquisitionfunction hyper-parameters θ after each batch itera-tion, as suggested by (Snoek et al., 2012). We stressthat this contrasts with common experimental prac-tice (Lyu et al., 2018), which sets θ to its maximumlikelihood value, and with the original algorithm pre-sentations (Desautels et al., 2014; Kandasamy et al.,2018; Gonzalez et al., 2016). In practice, marginal-ization yields a decrease in regret of up to three or-ders of magnitude (for the Cosines function), leadingto much stronger baselines. For ATS, we re-samplemultiple θs from their data posterior for each batchpoint, as explained in Section 3.1. For both approxi-mate marginalization and ATS, we perform the θ sam-pling by employing the Affine Invariant MCMC En-semble sampler provided by the emcee Python library(Foreman-Mackey et al., 2013). We decided to performall the experiments in the batch BO setting, ratherthan asynchronous, to keep the uncertainty constantand fixed across the algorithms and time. We remark

1Their hallucination technique might generalize, but weadhere to the common experimental practice.

Sampling Acquisition Functions for Batch Bayesian Optimization

0 20 40 60 80iteration

3

2

1sm

alle

st fu

nctio

n va

lue hartmann, M=10

ATS-EISeq. EI

0 20 40 60 80iteration

800

600

400eggholder, M=5

ATS-EISeq. EI

0 20 40 60iteration

1

4e-01

4branin, M=10

ATS-LCBSeq. LCB

Figure 1: Parallel vs. sequential BO on synthetic functions for varying batch size M and acquisition function.

that in practice ATS and the baselines can be easilyadapted to work in the asynchronous setting. Surro-gate models are always initialized with 5 random eval-uations (not shown).

4.1 Synthetic Functions

Here, we benchmark on a set of synthetic func-tions that are notoriously hard to optimize dueto the presence of multiple local or global minima(Eggholder, Cosines, Hartmann), or deep valley-likeregions (Branin, Rosenbrock). Therefore, they arewell-known in the global optimization community, andhave been previously employed as benchmarks in thecontext of batch BO (Kathuria et al., 2016; Gonzalezet al., 2016; Gupta et al., 2018; Kandasamy et al.,2018). More information on the functions can be foundin Table 2 in the Appendix.

Parallelizing a Sequential Acquisition Func-tion. Before comparing to the parallel BO baselines,we demonstrate that the diversity induced by sam-pling surrogate model hyper-parameters alone yieldsan efficient parallel BO algorithm. Figure 1 com-pares pure ATS to sequential BO, using LCB orEI, on three qualitatively different synthetic func-tions, plotting the smallest function value found un-til batch iteration t. Mean values and one standarderror over at least 10 repetitions are reported. ATSconsistently outperforms the sequential baseline andachieves close to ideal speedup on the high-dimensionaland multi-modal Hartmann function. The speedupson Eggholder and Branin (3.1× for M = 5 and 5× forM = 10, respectively) further highlight the benefits ofparallel BO and illustrate the effectiveness of ATS inparallelizing an arbitrary acquisition function.

ATS Variants vs. Parallel Baselines. Figures2 and 3 present our benchmark results, providing anexperimental comparison of the ATS variants to theparallel baselines: LP, B-LCB and P-TS. For LP andATS, we chose the acquisition function that showedthe best sequential behavior2 on the given synthetic

2For Hartmann6, where EI and LCB are on par, weselected EI, as it exhibited a better parallel behavior.

function. It is in fact well-known that the relative per-formance of acquisition functions depends on the opti-mization problem at hand (Hoffman et al., 2011). Theplots report mean values of regret (distance from thefunction minimum, clipped at 1e−6) over at least 10repetitions of the experiments; the ATS variant withthe best performance on the given task is highlightedwith a thicker line. In all plots but one, plain ATSeventually improves on P-TS, highlighting the bene-fits of focusing on the probability of maximizing anacquisition function rather than the model posterior.

We separate the benchmark functions into two cate-gories: Figure 2 presents results for valley-like func-tions. ATS on sequential methods performs compa-rably to P-TS and LP while B-LCB seems to excelon low-dimensional, relatively easy to optimize func-tions like Branin. The use of ATS to enhance existingparallel acquisition functions is successful in this set-ting: P-TS and B-LCB are almost always improvedupon, and enhanced versions are the best perform-ing methods. On the strongly multi-modal functionsshown in Figure 3 (Cosines and Eggholder exhibit si-nusoidal oscillations), instead, the ATS variants ap-plied to the sequential functions usually already out-perform the baselines, implying that the dependenceon hyper-parameters alone offers a competitive degreeof parallelism. An exception is the Hartmann func-tion, for which B-LCB performs particularly well, andits ATS enhancement outperforms all other methods.

2 4 6iteration

10 6

10 5

10 4

10 3

10 2

10 1

100

101

regr

et

branin, M=10

LP-LCBj-ATS-LCBATS-LCBB-LCBP-TS1p-ATS-P-TS1p-ATS-B-LCB

5 10 15iteration

102

103

104

105

rosenbrock, M=5LP-EIATS-EIB-LCBP-TS1p-ATS-P-TS1p-ATS-B-LCB

Figure 2: Benchmark on valley-like functions for varyingbatch size M and acquisition function.

A. De Palma, C. Mendler-Dunner, T. Parnell, A. Anghel, H. Pozidis

Table 1: Minima found by batch BO methods after a fixed number of iterations. The table reports mean values over atleast 10 repetitions and one standard error. Column title is structured as: synthetic function, number of batch iterations,batch size, acquisition function. ∗B-LCB and P-TS employ respectively LCB and TS as acquisition functions in all cases.

Algorithm Branin Cosines Hartmann Eggholder Rosenbrock7 it., M = 10, LCB 9 it., M = 5, EI 9 it., M = 10, EI 19 it., M = 5, EI 19 it., M = 5, EI

Sequential 1.0682± 0.1666 −1.5187± 0.1117 −1.8188± 0.2862 −745.4952± 45.2170 1630.9149± 352.1795ATS 0.3981± 0.0002 −1.77321± 6e−8 −3.2810± 0.0172 −878.4349± 17.5301 143.4917± 30.7108

LP 0.3994± 0.0012 −1.7727± 1e−5 −3.2196± 0.0293 −886.5274± 14.9021 176.5940± 42.9050B-LCB∗ 0.3979± 2e−6 −1.7731± 5e−5 −3.2898± 0.0144 −830.1668± 27.8264 66.4483± 12.6149

P-TS∗ 0.3994± 0.0003 −1.4832± 0.1476 −3.0904± 0.0261 −863.1183± 39.2462 86.7537± 28.9031ATS-B-LCB∗ 0.3979± 1e−6 −1.77315± 4e−5 −3.3064± 0.0109 −888.9844± 17.3145 125.8718± 37.7826

ATS-P-TS∗ 0.3993± 0.0003 −1.5798± 0.1289 −3.1295± 0.0202 −803.8516± 50.5051 36.6755± 6.3740

2 4 6 8iteration

10 6

10 5

10 4

10 3

10 2

10 1

100cosines, M=5

LP-EIj-ATS-EIATS-EIB-LCBP-TS1p-ATS-P-TS0.5p-ATS-B-LCB

2 4 6 8iteration

10 1

100

hartmann, M=10

LP-EIj-ATS-EIATS-EIB-LCBP-TS1p-ATS-P-TS0.5p-ATS-B-LCB

5 10 15iteration

102

102.60

regr

et

eggholder, M=5LP-EIATS-EIB-LCBh-ATS-EIP-TS1p-ATS-P-TS0.5p-ATS-B-LCB

Figure 3: Benchmark on multi-modal synthetic functions for varying batch size M .

For these objective functions, diversity seems to becrucial to avoid stagnating in one of the many min-ima. In fact, the variants of ATS presented in §3.2 arethe algorithms showing the best average performance.Based on our experiments, of all the analyzed variants,we would recommend the use of ATS-B-LCB or j-ATS.p = 0.5 is a good trade-off for using ATS on parallelacquisition functions in most use cases. A summaryof the optimization results at the last iteration of theplots in Figures 2 and 3, including the standard errorof the mean, can be found in Table 1.

Intra-batch analysis. In Figure 4, we turn to amore detailed analysis of the ATS variants by look-ing within the batches. As in Figure 1, we plot themean and its standard error for at least 10 repetitions;we show violin plots (where the white dot representsthe mean) for the distribution of the average `2 dis-tance of each batch points from the others, across therepetitions. The improvement of j-ATS on ATS andof the enhanced versions over the parallel baselines ismatched by an increase in batch diversity, visible fromthe higher mean in the violin plots, and by the lackof the lower tail in the distance distribution. The lat-ter effect is particularly important as it means thatclusters of nearby points (with distance close to 0) areremoved when using j-ATS on sequential acquisition

functions and ATS on the parallel ones. The improve-ment tends to be more visible in the first few and thelast few batch iterations.

4.2 Hyper-Parameter Optimization

We perform a similar comparison on the real-worldproblem of Hyper-Parameter Optimization (HPO). Wefocus on gradient boosting decision trees, as they enjoywidespread adoption in academia, industry and com-petitive data science due to their state-of-the-art per-formance in a wide variety of machine learning tasks(Ke et al., 2017). Specifically, our goal is to tunethe hyper-parameters of the popular XGBoost (Chenand Guestrin, 2016), so as to minimize the logisticloss incurred on a binary classification problem. Weutilized the well-known Higgs (P. Baldi and P. Sad-owski and D. Whiteson, 2014) dataset, freely availablewithin the UCI ML repository, whose task is to predictwhether a given process will produce Higgs bosons ornot. As the overall number of signals in the originaldataset is larger than ten millions, we uniformly sub-sampled ≈ 1′120′000 examples for the training, and≈ 400′000 for the validation set to make the experi-ments feasible on our infrastructure. We tuned 5 dif-ferent hyper-parameters; details can be found in Ap-pendix C. Figure 5 shows the minimum loss incurredon the fixed validation set after t batch iterations, av-

Sampling Acquisition Functions for Batch Bayesian Optimization

1 2 3 4 5 6 7 8 9

10 2

10 1

100

regr

et

hartmann, M=10B-LCB0.5p-ATS-B-LCB

1 2 3 4 5 6 7 8 9

10 1

100re

gret

hartmann, M=10ATS-EIj-ATS-EI

2.5 5.0 7.5 10.0 12.5 15.0 17.5iteration

102

103

104

105

regr

et

rosenbrock, M=5P-TS1p-ATS-P-TS

1 2 3 4 5 6 7 8 90

1

dist

ance

hartmann, M=10B-LCB0.5p-ATS-B-LCB

1 3 5 7 9 11 13 15 17 19iteration

5

10

15

dist

ance

rosenbrock, M=5P-TS1p-ATS-P-TS

1 2 3 4 5 6 7 8 90

1

2

dist

ance

hartmann, M=10ATSj-ATS

Figure 4: Comparison of techniques for batch diversity: regret and intra-batch average `2 distance.

eraged over 5 repetitions. We opted for a large batchsize (M = 20) to mimic a realistic scenario where thesignificant time cost of HPO needs to be amortizedthrough parallelism. Moreover, we chose LCB overEI due to its better sequential performance, as in theprevious section. The results show that ATS variantsapplied to sequential acquisition functions are compet-itive to B-LCB also on HPO, while LP shows inferiorperformance. The ATS variant exhibiting the best be-havior is, again, j-ATS, which achieves almost idealspeedup to sequential BO. This is to be expected, asin larger batches the diversity provided by samplingθ alone might not be sufficient, and employing hallu-cinations might be short-sighted with respect to thefuture model updates.

5 Conclusions and Future Work

We presented Acquisition Thompson Sampling (ATS),a novel technique for parallel Bayesian Optimization(BO) based on sampling acquisition functions from astochastic process through their dependency on thesurrogate model hyper-parameters. ATS can be suc-cessfully used to parallelize sequential acquisition func-tions or to make parallel BO methods scale further.We enhanced this conceptually simple algorithm bytranslating ideas from the batch BO literature intoour randomized framework: First (j-ATS), by addingjitter to the acquisition functions, we sample differentexplore-exploit trade-offs from a prior distribution em-bedded in the definition of the process, as opposed toobtaining these trade-offs by solving a multi-objectiveoptimization problem (Gupta et al., 2018). Second (h-ATS), we employ hallucinations of the pending evalu-ations to diversify the data posterior of the surrogate

model hyper-parameters, rather than to update thesurrogate model itself (Desautels et al., 2014). We ob-tain a versatile and multi-faceted algorithm, usable toparallelize any sequential acquisition function.

We benchmarked against two state-of-the-art batchBO algorithms (B-LCB, LP) (Gonzalez et al., 2016;Desautels et al., 2014) and against Parallel ThompsonSampling (Kandasamy et al., 2018) on five syntheticfunctions from the global optimization literature, andon tuning a popular gradient boosting tree algorithm.While ATS should be seen as orthogonal to the ex-isting parallel methods, as it can be employed to in-crease their iteration efficiency, we show that the vari-ability induced by sampling surrogate model hyper-parameters suffices to create enough parallelism tobe competitive with, or to outperform, the employedparallel baselines on strongly multi-modal functions.Moreover, enhancing B-LCB and P-TS is particularlyeffective on objective functions exhibiting deep valleys.

100 101

iteration

0.4975

0.5000

0.5025

0.5050

0.5075

0.5100

0.5125

0.5150

0.5175

Logi

stic

loss

HPO, XGBoost on Higgs, M=20LP-LCBj-ATS-LCBATS-LCBB-LCBh-ATS-LCBSequential LCB

Figure 5: Hyper-parameter tuning of XGBoost on theHiggs dataset, with batch size M = 20.

A. De Palma, C. Mendler-Dunner, T. Parnell, A. Anghel, H. Pozidis

References

Felix Berkenkamp, Angela P. Schoellig, and AndreasKrause. No-regret bayesian optimization with un-known hyperparameters. Journal of Machine Learn-ing Research, 20(50):1–24, 2019.

Adam D. Bull. Convergence rates of efficient globaloptimization algorithms. JMLR, 12:2879–2904,November 2011. ISSN 1532-4435.

Olivier Chapelle and Lihong Li. An empirical evalu-ation of thompson sampling. In Proceedings of the24th International Conference on Neural Informa-tion Processing Systems, pages 2249–2257, 2011.

Tianqi Chen and Carlos Guestrin. Xgboost: A scal-able tree boosting system. In Proceedings of the22Nd ACM SIGKDD International Conference onKnowledge Discovery and Data Mining, pages 785–794, 2016.

Clement Chevalier and David Ginsbourger. Fast com-putation of the multi-points expected improvementwith applications in batch selection. In Revised Se-lected Papers of the 7th International Conferenceon Learning and Intelligent Optimization - Volume7997, pages 59–69, 2013.

Sayak Ray Chowdhury and Aditya Gopalan. On ker-nelized multi-armed bandits. In Proceedings of the34th International Conference on Machine Learn-ing, volume 70, pages 844–853, 2017.

Erik A. Daxberger and Bryan Kian Hsiang Low. Dis-tributed batch Gaussian process optimization. InProceedings of the 34th International Conference onMachine Learning, volume 70, pages 951–960, 2017.

Thomas Desautels, Andreas Krause, and Joel Bur-dick. Parallelizing exploration-exploitation trade-offs with gaussian process bandit optimization. InProc. International Conference on Machine Learn-ing (ICML), 2014.

D. Foreman-Mackey, D. W. Hogg, D. Lang, andJ. Goodman. emcee: The mcmc hammer. PASP,125:306–312, 2013.

J. Gonzalez, Z. Dai, P. Hennig, and N. Lawrence.Batch bayesian optimization via local penalization.In Proceedings of the 19th International Conferenceon Artificial Intelligence and Statistics (AISTATS2016), volume 51, pages 648–657, 2016.

Jonathan Goodman and Jonathan Weare. Ensemblesamplers with affine invariance. 5(1):65–80, 2010.doi: 10.2140/camcos.2010.5.65.

Sunil Gupta, Alistair Shilton, Santu Rana, and SvethaVenkatesh. Exploiting strategy-space diversity forbatch bayesian optimization. In Proceedings of theTwenty-First International Conference on Artificial

Intelligence and Statistics, volume 84, pages 538–547, 2018.

Jose Miguel Hernandez-Lobato, Michael Gelbart,Matthew Hoffman, Ryan Adams, and ZoubinGhahramani. Predictive entropy search for bayesianoptimization with unknown constraints. In Proceed-ings of the 32nd International Conference on Ma-chine Learning, volume 37, pages 1699–1707, 2015.

Matthew Hoffman, Eric Brochu, and Nando de Fre-itas. Portfolio allocation for bayesian optimization.In Proceedings of the Twenty-Seventh Conference onUncertainty in Artificial Intelligence, UAI’11, pages327–336, 2011.

Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimization forgeneral algorithm configuration. In Proceedings ofthe 5th International Conference on Learning andIntelligent Optimization, LION’05, pages 507–523,2011.

Donald R. Jones, Matthias Schonlau, and William J.Welch. Efficient global optimization of expensiveblack-box functions. J. Global Optimization, 13(4):455–492, 1998.

Kirthevasan Kandasamy, Akshay Krishnamurthy, JeffSchneider, and Barnabas Poczos. Parallelisedbayesian optimisation via thompson sampling. InProceedings of the Twenty-First International Con-ference on Artificial Intelligence and Statistics, vol-ume 84, pages 133–142, 2018.

Tarun Kathuria, Amit Deshpande, and PushmeetKohli. Batched gaussian process bandit optimiza-tion via determinantal point processes. In Advancesin Neural Information Processing Systems 29, pages4206–4214. 2016.

Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang,Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu.LightGBM: A highly efficient gradient boosting de-cision tree. In NIPS, pages 3149–3157, 2017.

Cheng Li, David Rubın de Celis Leal, Santu Rana,Sunil Gupta, Alessandra Sutti, Stewart Greenhill,Teo Slezak, Murray Height, and Svetha Venkatesh.Rapid bayesian optimisation for synthesis of shortpolymer fiber materials. Scientific Reports, 7(1):5683, 2017.

Wenlong Lyu, Fan Yang, Changhao Yan, Dian Zhou,and Xuan Zeng. Batch Bayesian optimization viamulti-objective acquisition ensemble for automatedanalog circuit design. In Proceedings of the 35th In-ternational Conference on Machine Learning, vol-ume 80, pages 3312–3320, 2018.

P. Baldi and P. Sadowski and D. Whiteson. Search-ing for Exotic Particles in High-energy Physics with

Sampling Acquisition Functions for Batch Bayesian Optimization

Deep Learning. Nature Communications 5:4308 doi:10.1038/ncomms5308, 2014.

CE. Rasmussen and CKI. Williams. Gaussian Pro-cesses for Machine Learning. Adaptive Computa-tion and Machine Learning. MIT Press, 2006.

Amar Shah and Zoubin Ghahramani. Parallel predic-tive entropy search for batch global optimization ofexpensive objective functions. In Advances in Neu-ral Information Processing Systems 28, pages 3330–3338. 2015.

Bobak Shahriari, Alexandre Bouchard-Cote, andNando Freitas. Unbounded bayesian optimizationvia regularization. In Proceedings of the 19th In-ternational Conference on Artificial Intelligence andStatistics, volume 51, pages 1168–1176, 2016a.

Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P.Adams, and Nando de Freitas. Taking the humanout of the loop: A review of bayesian optimization.Proceedings of the IEEE, 104(1):148–175, 2016b.

Jasper Snoek, Hugo Larochelle, and Ryan P Adams.Practical bayesian optimization of machine learningalgorithms. In Advances in Neural Information Pro-cessing Systems 25, pages 2951–2959. 2012.

Benjamin Solnik, Daniel Golovin, Greg Kochanski,John Elliot Karro, Subhodeep Moitra, and D. Scul-ley. Bayesian optimization for a better dessert.In Proceedings of the 2017 NIPS Workshop onBayesian Optimization, 2017.

Jost Tobias Springenberg, Aaron Klein, StefanFalkner, and Frank Hutter. Bayesian optimizationwith robust bayesian neural networks. In Advancesin Neural Information Processing Systems 29, pages4134–4142. 2016.

Niranjan Srinivas, Andreas Krause, Sham M. Kakade,and Matthias W. Seeger. Information-theoretic re-gret bounds for gaussian process optimization in thebandit setting. information theory. IEEE Transac-tions on, page 3265, 2012.

the GPyOpt authors. GPyOpt: A bayesian optimiza-tion framework in python. http://github.com/

SheffieldML/GPyOpt, 2016.

W.R. Thompson. On the likelihood that one unknownprobability exceeds another in view of the evidenceof two samples. Biometrika, 1933.

J. Wang, S. C. Clark, E. Liu, and P. I. Frazier. ParallelBayesian Global Optimization of Expensive Func-tions. ArXiv e-prints, February 2016.

Jian Wu and Peter Frazier. The parallel knowledgegradient method for batch bayesian optimization. InAdvances in Neural Information Processing Systems29, pages 3126–3134. 2016.

A. De Palma, C. Mendler-Dunner, T. Parnell, A. Anghel, H. Pozidis

A Batch Diversity in ATS

This Appendix provides additional details on the twotechniques, discussed in Section 3.2, for achievingbatch diversity within the ATS framework.

A.1 j-ATS

The pseudo-code for a batch iteration of the j-ATSvariant of our algorithm is given in Algorithm 2.

Algorithm 2 Batch iteration, jittered AcquisitionThompson Sampling (j-ATS).

input dataset Dt, sequential acquisition functiona(x,θ|Dt), batch size M .

1: for i = 1, . . . ,M in parallel do2: sample s new GP hyper-parameter vectors ac-

cording to θq ∼ p(θ|Dt).

3: sample the jitter according to j ∼ p(j).4: as,j(x|Dt)← 1

s

∑sq=1 a(x,θq, j|Dt).

5: xt+i ← arg maxX as,j(x|Dt), the i-th batchpoint.

6: evaluate f(xt+i).

7: Dt+M ← Dt ∪ {(xk, f(xk)) ∀ t < k ≤ t+M} .

A.2 h-ATS

As explained in Section 2.2, approaches for parallel BOlike (Desautels et al., 2014) work by augmenting thedataset with hallucinations of the pending evaluations.Evaluation i is then selected by having a(x) make useof the hallucinated surrogate model N (µ,K | Di−1)with Di−1 = Dt ∪ {(x′k, hk) ∀ t < k ≤ i− 1} , wheret is the time of the last completed evaluation and hkdenotes a hallucination of f(x′k). As the hk’s are ap-proximations resulting from the last updated surro-gate model, these approaches might be over-confidenton the collected observations (Kathuria et al., 2016).

If we perform the sampling of the acquisition functionsin a sequential manner, we can exploit the same tech-nique in our randomized framework by hallucinatingthe data posterior, rather than the surrogate model(which remains independent of the pending evalua-tions). By doing that, we can make use of the knowl-edge of the locations of the pending evaluations in or-der to ”update” the stochastic process to as(x|Di−1).This means that, at iteration i, we sample:

ahs (x) =1

s

s∑q=1

a(x;θq) s.t. θq ∼ p(θ|Di−1). (5)

From the high-level perspective, sampling from thehallucinated data posterior means sampling models

Algorithm 3 Batch iteration, hallucinated Acquisi-tion Thompson Sampling (h-ATS).

input dataset Dt, sequential acquisition functiona(x,θ|Dt), batch size M .

1: Dt := Dt.2: for i = 1, . . . ,M do3: sample s new GP hyper-parameter vectors ac-

cording to θq ∼ p(θ|Dt+i−1), the hallucinateddata posterior.

4: ahs (x|Dt)← 1s

∑sq=1 a(x,θq|Dt).

5: xt+i ← arg maxX ahs (x|Dt), the i-th batch

point.

6: ht+i = 1s

∑sq=1 µ(xt+i,θq|Dt).

7: Dt+i ← Dt+i−1 ∪ {(xt+i, ht+i)}.8: for i = 1, . . . ,M in parallel do9: evaluate f(xt+i).

10: Dt+M ← Dt ∪ {(xk, f(xk)) ∀ t < k ≤ t+M}.

that should explain the hallucinations. In fact, the dif-ference between p(θ|Dt+i−1) and p(θ|Dt) lies in thatthe former takes the hallucinations into account in itsdata likelihood p(Dt ∪ {(x′k, hk) ∀ t < k ≤ i− 1}|θ).The practical effect, analogously to j-ATS, is to in-crease the diversity across the batch points. In thiscase, though, no acquisition function-specific jitterprior needs to be devised; the approach is thereforemore readily applicable to new acquisition functions.For what concerns hallucinations, we adapt the ap-proximation employed by Desautels et al. (2014) tomake use of the θq samples: hi = 1

s

∑sq=1 µ(x′i;θq|Dt).

We point out that the θq vectors are those sampled us-ing the hallucinated data posterior, as in (5), but theposterior is on the last updated model (without hal-lucinations). We summarize a batch iteration of thishallucinated ATS (h-ATS) in Algorithm 3.

B Data posterior sampling andimplementation details

All ATS variants sample acquisition functions froma process defined through the hyper-parameters dataposterior. As outlined in Algorithms 1 and 3, for ATSand h-ATS, it suffices to sample θ, the surrogate modelhyper-parameters, to obtain an acquisition function(Equation (3)). θ consists of θk, the GP kernel hyper-parameters, and a constant prior mean µp (cf. Section2.1). Making use of the Bayes rule, we can write:

p(θ|Dt) =1

Zp(Dt|θ) p(θ),

where 1Z is a normalization constant. p(Dt|θ) is

the marginal likelihood of the collected data un-der the Gaussian process (Rasmussen and Williams,

Sampling Acquisition Functions for Batch Bayesian Optimization

f(x) d X minX f(x)

Branin 2 [−5, 10]× [0, 15] 0.3979Cosines 2 [0, 1]2 −1.773Hartmann6 6 [0, 1]6 −3.322Eggholder 2 [−512, 512]2 −959.64Rosenbrock4 4 [−5, 10]4 0

Table 2: Synthetic functions characteristics.

2006), while p(θ) is the prior over the model hyper-parameters. For what concerns the prior, we exploreddifferent options and eventually chose a modificationof that employed by the GPyOpt authors (2016), asit was the one that performed the best in a variety ofcases. Specifically:

p(θk) = Γ(α = 1, β = 6) (6)

p(µp) = Uniform(−3, 3). (7)

The priors are independent as in Snoek et al. (2012).As we found the choice of the prior to be stronglydependent on the output range of f , we employ z-normalization of the outputs {yi} after each modelupdate.

To sample from p(θ|Dt), we employ Affine InvariantMCMC Ensemble sampling by Goodman and Weare(2010). Its computational cost is comparable to thatof single-particle algorithms like Metropolis sampling(Goodman and Weare, 2010). For BO, the main bot-tleneck in evaluating p(θ|Dt) resides in the matrix in-version to compute the GP marginal likelihood, cu-bic in the size of the samples (Snoek et al., 2012).Our parallel algorithm hence does not add any com-putational cost to a sequential Bayesian treatment ofhyper-parameters: it suffices to retain more θ samples.

For j-ATS, the hyper-parameters also include the jitterj, a parameter of the acquisition function itself. Thedata posterior defining the process is hence p(θ, j|Dt).As the data likelihood is independent of the param-eters that do not belong to the surrogate model, weonly need to define the prior of j. We then samplefrom this prior independently from the θ sampling (seeAlgorithm 2). Let C be a Bernoulli random variablewith p = 0.5. Then:

EI: j|C = 1 ∼ logUniform(−3, 0); (8)

LCB: j|C = 1 ∼ Beta(1, 12) (9)

while we revert to ATS, as presented in Section 3.1(without jitter), if C = 0. Reverting to ATS meansthat j = 0 (no exploration incentive) for EI, and j = 1for LCB. In the latter case, we treated j as a hyper-parameter to be tuned and employed the same valuefor B-LCB.

Hyper-parameter domain

Number of boosting rounds [16, 256]Step shrinkage (η) [0.1, 1]Maximum depth of a tree [2, 8]Regularization coefficient (λ) [10−2, 107]Subsample ratio of columns [0.1, 1]

Table 3: Hyper-parameter domains for the tuning of XG-Boost.

C Experiments details

Finally, we provide additional details for the functionswe optimize in the experiments presented in Section 4.Table 2 summarizes the characteristics of the employedsynthetic functions.

Table 3 reports the domains for the 5 XGBoost hyper-parameters we tuned for the experiments in Section4.2; we kept the maximum number of bins equal to 64.