Continuous-time birth-death mcmc for bayesian regression tree models

被引：0

作者：

Mohammadi, Reza ^{[1
]}

Pratola, Matthew ^{[2
]}

Kaptein, Maurits ^{[3
]}

机构：

[1] Amsterdam Business School, University of Amsterdam, Amsterdam, Netherlands

[2] Department of Statistics, Ohio State University, OH, United States

[3] Statistics and Research Methods, University of Tilburg, Tilburg, Netherlands

来源：

Journal of Machine Learning Research | 2020年 / 21卷

关键词：

Bayesian networks - Computational efficiency - Continuous time systems - Learning algorithms - Regression analysis - Mixing - Markov processes;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

Decision trees are flexible models that are well suited for many statistical regression problems. In the Bayesian framework for regression trees, Markov Chain Monte Carlo (MCMC) search algorithms are required to generate samples of tree models according to their posterior probabilities. The critical component of such MCMC algorithms is to construct goodMetropolis-Hastings steps to update the tree topology. Such algorithms frequently suffer from poor mixing and local mode stickiness; therefore, the algorithms are slow to converge. Hitherto, authors have primarily used discrete-time birth/death mechanisms for Bayesian (sums of) regression tree models to explore the tree-model space. These algorithms are efficient, in terms of computation and convergence, only if the rejection rate is low which is not always the case. We overcome this issue by developing a novel search algorithm which is based on a continuous-time birth-death Markov process. The search algorithm explores the tree-model space by jumping between parameter spaces corresponding to different tree structures. The jumps occur in continuous time corresponding to the birth-death events which are modeled as independent Poisson processes. In the proposed algorithm, the moves between models are always accepted which can dramatically improve the convergence and mixing properties of the search algorithm. We provide theoretical support of the algorithm for Bayesian regression tree models and demonstrate its performance in a simulated example. ©2020 Reza Mohammadi, Matthew Pratola and Maurits Kaptein.

引用