Mean field inference for the Dirichlet process mixture model

被引:5
作者
Zobay, O. [1 ]
机构
[1] Univ Bristol, Dept Math, Bristol BS8 1TW, Avon, England
基金
英国工程与自然科学研究理事会;
关键词
Bayesian nonparametrics; approximation methods; variational inference; density estimation; DENSITY-ESTIMATION; SAMPLING METHODS;
D O I
10.1214/08-EJS339
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
We present a systematic study of several recently proposed methods of mean field inference for the Dirichlet process mixture (DPM) model. These methods provide approximations to the posterior distribution and are derived using the truncated stick-breaking representation and related approaches. We investigate their use in density estimation and cluster allocation and compare to Monte-Carlo results. Further, more specific topics include the general mathematical structure of the mean field approximation, the handling of the truncation level, the effect of including a prior on the concentration parameter a of the DPM model, the relationship between the proposed variants of the mean field approximation, and the on to Maxi mum a-posteriori estimation of the DPM model.
引用
收藏
页码:507 / 545
页数:39
相关论文
共 28 条
[1]  
[Anonymous], 2001, ADV MEAN FIELD METHO
[2]   MIXTURES OF DIRICHLET PROCESSES WITH APPLICATIONS TO BAYESIAN NONPARAMETRIC PROBLEMS [J].
ANTONIAK, CE .
ANNALS OF STATISTICS, 1974, 2 (06) :1152-1174
[3]  
Bishop C. M., 2009, Pattern Recognition and Machine Learning
[4]   Variational Inference for Dirichlet Process Mixtures [J].
Blei, David M. ;
Jordan, Michael I. .
BAYESIAN ANALYSIS, 2006, 1 (01) :121-143
[5]   Latent Dirichlet allocation [J].
Blei, DM ;
Ng, AY ;
Jordan, MI .
JOURNAL OF MACHINE LEARNING RESEARCH, 2003, 3 (4-5) :993-1022
[6]   BAYESIAN DENSITY-ESTIMATION AND INFERENCE USING MIXTURES [J].
ESCOBAR, MD ;
WEST, M .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 1995, 90 (430) :577-588
[7]   ESTIMATING NORMAL MEANS WITH A DIRICHLET PROCESS PRIOR [J].
ESCOBAR, MD .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 1994, 89 (425) :268-277
[8]  
Escobar Michael David, 1988, Estimating the means of several normal populations by nonparametric estimation of the distribution of the means
[9]   BAYESIAN ANALYSIS OF SOME NONPARAMETRIC PROBLEMS [J].
FERGUSON, TS .
ANNALS OF STATISTICS, 1973, 1 (02) :209-230
[10]   Gibbs sampling methods for stick-breaking priors [J].
Ishwaran, H ;
James, LF .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2001, 96 (453) :161-173