New Sampling Strategies for Bayesian Optimization of Functions with Location-Dependent Sample Noise

Abstract

The problem of finding the global extremum of noisy, black-box functions has applications in many different fields, and it has been studied extensively in the literature. Bayesian optimization is a technique for solving this problem where a set of samples of the objective function, collected by the optimizer, is used to infer a probability distribution over possible values of the objective function, which is used to estimate its global maximum or minimum. One essential component of the Bayesian optimization approach is the strategy for collecting samples, which sequentially picks new sample points based on the current distribution over the function’s possible values. Although there are many cases where the noise associated with a sample value depends greatly on the location of the sample point, no sampling strategies have been developed to take this information into account if it is known to the optimizer. In this paper, we present two sampling strategies for Bayesian optimization in the case of location-dependent sample noise, and show that they outperform two of the most commonly used sampling strategies on randomly-generated objective functions.