Adaptive strategy for stratified Monte Carlo sampling

We consider the problem of stratified sampling for Monte Carlo integration of a random variable. We model this problem in a K-armed bandit, where the arms represent the K strata. The goal is to estimate the integral mean, that is a weighted average of the mean values of the arms. The learner is allowed to sample the variable n times, but it can decide on-line which stratum to sample next. We propose an UCB-type strategy that samples the arms according to an upper bound on their estimated standard deviations. We compare its performance to an ideal sample allocation that knows the standard deviations of the arms. For sub-Gaussian arm distributions, we provide bounds on the total regret: a distributiondependent bound of order poly(λmin-1)O (n-3,2)1 that depends on a measure of the disparity λmin of the per stratum variances and a distribution-free bound poly(K)O (n-7/6) that does not. We give similar, but somewhat sharper bounds on a proxy of the regret. The problemindependent bound for this proxy matches its recent minimax lower bound in terms of n up to a log n factor. © 2015 Alexandra Carpentier, Remi Munos and András Antos.