Generalized Guerra's interpolation schemes for dense associative neural networks

被引:26
作者
Agliari, Elena [1 ]
Alemanno, Francesco [2 ,3 ]
Barra, Adriano [2 ,4 ]
Fachechi, Alberto [2 ,4 ]
机构
[1] Dipartimento Matemat Guido Castelnuovo, Rome, Italy
[2] Univ Salento, Dipartimento Matemat & Fis Ennio De Giorgi, Lecce, Italy
[3] CNR Nanotec Lecce, Lecce, Italy
[4] INFN, Sez Lecce, Lecce, Italy
关键词
Associative neural networks; Statistical mechanics; PDE-theory; Hebbian learning; Pattern recognition; STATISTICAL-MECHANICS; HOPFIELD MODEL; GIBBS-STATES; PATTERNS; REGIME;
D O I
10.1016/j.neunet.2020.05.009
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this work we develop analytical techniques to investigate a broad class of associative neural networks set in the high-storage regime. These techniques translate the original statistical-mechanical problem into an analytical-mechanical one which implies solving a set of partial differential equations, rather than tackling the canonical probabilistic route. We test the method on the classical Hopfield model - where the cost function includes only two-body interactions (i.e., quadratic terms) - and on the "relativistic"Hopfield model - where the (expansion of the) cost function includes p-body (i.e., of degree p) contributions. Under the replica symmetric assumption, we paint the phase diagrams of these models by obtaining the explicit expression of their free energy as a function of the model parameters (i.e., noise level and memory storage). Further, since for non-pairwise models ergodicity breaking is non necessarily a critical phenomenon, we develop a fluctuation analysis and find that criticality is preserved in the relativistic model. (C) 2020 Elsevier Ltd. All rights reserved.
引用
收藏
页码:254 / 267
页数:14
相关论文
共 45 条
[1]  
Agliari E., 2020, J MATH PHYS UNPUB
[2]   Dreaming neural networks: rigorous results [J].
Agliari, Elena ;
Alemanno, Francesco ;
Barra, Adriano ;
Fachechi, Alberto .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2019,
[3]   The relativistic Hopfield network: Rigorous results [J].
Agliari, Elena ;
Barra, Adriano ;
Notarnicola, Matteo .
JOURNAL OF MATHEMATICAL PHYSICS, 2019, 60 (03)
[4]   Free energies of Boltzmann machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit [J].
Agliari, Elena ;
Barra, Adriano ;
Tirozzi, Brunello .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2019,
[5]   Neural Networks Retrieving Boolean Patterns in a Sea of Gaussian Ones [J].
Agliari, Elena ;
Barra, Adriano ;
Longo, Chiara ;
Tantari, Daniele .
JOURNAL OF STATISTICAL PHYSICS, 2017, 168 (05) :1085-1104
[6]   SPIN-GLASS MODELS OF NEURAL NETWORKS [J].
AMIT, DJ ;
GUTFREUND, H .
PHYSICAL REVIEW A, 1985, 32 (02) :1007-1018
[7]   STORING INFINITE NUMBERS OF PATTERNS IN A SPIN-GLASS MODEL OF NEURAL NETWORKS [J].
AMIT, DJ ;
GUTFREUND, H ;
SOMPOLINSKY, H .
PHYSICAL REVIEW LETTERS, 1985, 55 (14) :1530-1533
[8]  
Amit DJ., 1989, Modeling Brain Function: The World of Attractor Neural Networks
[9]  
[Anonymous], 2008, MACH LEARN P 25 INT
[10]  
[Anonymous], 2009, ARTIF INTELL