Distributionally Robust Bayesian Optimization with φ-divergences

被引:0
作者
Husain, Hisham [1 ]
Vu Nguyen [1 ]
van den Hengel, Anton [1 ]
机构
[1] Amazon, Seattle, WA 98109 USA
来源
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023) | 2023年
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. [26], which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in phi-divergences, which subsumes many popular choices, such as the chi(2)-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] On distributionally robust multiperiod stochastic optimization
    Analui, Bita
    Pflug, Georg Ch.
    COMPUTATIONAL MANAGEMENT SCIENCE, 2014, 11 (03) : 197 - 220
  • [22] Distributionally robust possibilistic optimization problems
    Guillaume, Romain
    Kasperski, Adam
    Zielinski, Pawel
    FUZZY SETS AND SYSTEMS, 2023, 454 : 56 - 73
  • [23] Distributionally Robust Optimization in Possibilistic Setting
    Guillaume, Romain
    Kasperski, Adam
    Zielinski, Pawel
    IEEE CIS INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS 2021 (FUZZ-IEEE), 2021,
  • [24] Distributionally Robust Optimization with Markovian Data
    Li, Mengmeng
    Sutter, Tobias
    Kuhn, Daniel
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
  • [25] A framework of distributionally robust possibilistic optimization
    Romain Guillaume
    Adam Kasperski
    Paweł Zieliński
    Fuzzy Optimization and Decision Making, 2024, 23 : 253 - 278
  • [26] Distributionally Robust Optimization with Data Geometry
    Liu, Jiashuo
    Wu, Jiayun
    Li, Bo
    Cui, Peng
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [27] Distributionally Robust Skeleton Learning of Discrete Bayesian Networks
    Li, Yeshu
    Ziebart, Brian D.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [28] Bridging Bayesian and Minimax Mean Square Error Estimation via Wasserstein Distributionally Robust Optimization
    Viet Anh Nguyen
    Shafieezadeh-Abadeh, Soroosh
    Kuhn, Daniel
    Esfahani, Peyman Mohajerin
    MATHEMATICS OF OPERATIONS RESEARCH, 2023, 48 (01) : 1 - 37
  • [29] Globalized distributionally robust optimization based on samples
    Yueyao Li
    Wenxun Xing
    Journal of Global Optimization, 2024, 88 : 871 - 900
  • [30] Distributionally Robust Optimization Under Distorted Expectations
    Cai, Jun
    Li, Jonathan Yu-Meng
    Mao, Tiantian
    OPERATIONS RESEARCH, 2023, : 969 - 985