Ising distribution as a latent variable model

被引:3
作者
Wohrer, Adrien [1 ]
机构
[1] Univ Clermont Auvergne, CNRS, SIGMA Clermont, Inst Pascal, F-63000 Clermont Ferrand, France
关键词
MEAN-FIELD THEORY; NEURAL-NETWORKS; SOLVABLE MODEL; SPIN; GENERATION; EFFICIENT;
D O I
10.1103/PhysRevE.99.042147
中图分类号
O35 [流体力学]; O53 [等离子体物理学];
学科分类号
070204 ; 080103 ; 080704 ;
摘要
During the past decades, the Ising distribution has attracted interest in many applied disciplines, as the maximum entropy distribution associated to any set of correlated binary ("spin") variables with observed means and covariances. However, numerically speaking, the Ising distribution is unpractical, so alternative models are often preferred to handle correlated binary data. One popular alternative, especially in life sciences, is the Cox distribution (or the closely related dichotomized Gaussian distribution and log-normal Cox point process), where the spins are generated independently conditioned on the drawing of a latent variable with a multivariate normal distribution. This article explores the conditions for a principled replacement of the Ising distribution by a Cox distribution. It shows that the Ising distribution itself can be treated as a latent variable model, and it explores when this latent variable has a quasi-normal distribution. A variational approach to this question reveals a formal link with classic mean-field methods, especially Opper and Winther's adaptive TAP approximation. This link is confirmed by weak coupling (Plefka) expansions of the different approximations and then by numerical tests. Overall, this study suggests that an Ising distribution can be replaced by a Cox distribution in practical applications, precisely when its parameters lie in the "mean-field domain."
引用
收藏
页数:17
相关论文
共 42 条
[1]  
Ackley David H, 1987, READINGS COMPUTER VI, P522, DOI [DOI 10.1016/B978-0-08-051581-6.50053-2, 10.1016/B978-0-08-051581, DOI 10.1016/B978-0-08-051581]
[2]   Synchronous firing and higher-order interactions in neuron pool [J].
Amari, S ;
Nakahara, H ;
Wu, S ;
Sakai, Y .
NEURAL COMPUTATION, 2003, 15 (01) :127-142
[3]  
BISHOP C. M., 2006, Pattern recognition and machine learning, DOI [DOI 10.1117/1.2819119, 10.1007/978-0-387-45528-0]
[4]  
Boyd Stephen, 2004, Convex Optimization, DOI 10.1017/CBO9780511804441
[5]  
Brette R, 2009, NEURAL COMPUT, V21, P188, DOI 10.1162/neco.2008.12-07-657
[6]   Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data [J].
Cocco, S. ;
Monasson, R. .
PHYSICAL REVIEW LETTERS, 2011, 106 (09)
[7]  
COX DR, 1955, J ROY STAT SOC B, V17, P129
[8]  
COX DR, 1972, J ROY STAT SOC C-APP, V21, P113, DOI 10.2307/2346482
[9]   On some models for multivariate binary variables parallel in complexity with the multivariate Gaussian distribution [J].
Cox, DR ;
Wermuth, N .
BIOMETRIKA, 2002, 89 (02) :462-469
[10]  
COX DR, 1958, J R STAT SOC B, V20, P215