Applications of hybrid Monte Carlo to Bayesian generalized linear models: Quasicomplete separation and neural networks

被引:22
作者
Ishwaran, H [1 ]
机构
[1] Cleveland Clin Fdn, Dept Biostat & Epidemiol Wb4, Cleveland, OH 44195 USA
关键词
Bayesian hierarchical models; feedforward neural networks; leapfrog algorithm; Markov chain Monte Carlo; random walk Metropolis-Hastings;
D O I
10.2307/1390827
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
The "leapfrog" hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis-Hastings. However, this entails very little additional programming work.
引用
收藏
页码:779 / 799
页数:21
相关论文
共 32 条
[1]  
Agresti A., 1996, An introduction to categorical data analysis
[2]  
ALBERT A, 1984, BIOMETRIKA, V71, P1
[3]  
Brockmann HJ, 1996, ETHOLOGY, V102, P1
[4]  
Cybenko G., 1989, Mathematics of Control, Signals, and Systems, V2, P303, DOI 10.1007/BF02551274
[5]  
Daniels MJ, 1998, COMP SCI STAT, V30, P192
[6]   BAYESIAN-INFERENCE FOR GENERALIZED LINEAR AND PROPORTIONAL HAZARDS MODELS VIA GIBBS SAMPLING [J].
DELLAPORTAS, P ;
SMITH, AFM .
APPLIED STATISTICS-JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES C, 1993, 42 (03) :443-459
[7]   THE THEORY OF HYBRID STOCHASTIC ALGORITHMS [J].
DUANE, S ;
KOGUT, JB .
NUCLEAR PHYSICS B, 1986, 275 (03) :398-420
[8]   HYBRID MONTE-CARLO [J].
DUANE, S ;
KENNEDY, AD ;
PENDLETON, BJ ;
ROWETH, D .
PHYSICS LETTERS B, 1987, 195 (02) :216-222
[10]   ON THE APPROXIMATE REALIZATION OF CONTINUOUS-MAPPINGS BY NEURAL NETWORKS [J].
FUNAHASHI, K .
NEURAL NETWORKS, 1989, 2 (03) :183-192