Combinatorial optimization by weight annealing in memristive hopfield networks

被引:16
作者
Fahimi, Z. [1 ]
Mahmoodi, M. R. [1 ]
Nili, H. [1 ]
Polishchuk, Valentin [2 ]
Strukov, D. B. [1 ]
机构
[1] UC Santa Barbara, Santa Barbara, CA 93106 USA
[2] Linkoping Univ, S-60174 Norrkoping, Sweden
关键词
NEURAL-NETWORKS;
D O I
10.1038/s41598-020-78944-5
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the recent demonstrations of efficient mixed-signal implementation based on emerging non-volatile memory devices. Such mixed-signal accelerators also enable very efficient implementation of various annealing techniques, which are essential for finding optimal solutions. Here we propose a "weight annealing" approach, whose main idea is to ease convergence to the global minima by keeping the network close to its ground state. This is achieved by initially setting all synaptic weights to zero, thus ensuring a quick transition of the Hopfield network to its trivial global minima state and then gradually introducing weights during the annealing process. The extensive numerical simulations show that our approach leads to a better, on average, solutions for several representative combinatorial problems compared to prior Hopfield neural network solvers with chaotic or stochastic annealing. As a proof of concept, a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem are solved experimentally using mixed-signal circuits based on, correspondingly, a 20 x 20 analog-grade TiO2 memristive crossbar and a 12 x 10 eFlash memory array.
引用
收藏
页数:10
相关论文
共 50 条
[41]   Stability of Hopfield-type neural networks II [J].
Xiaoxin Liao ;
Yang Liao .
Science in China Series A: Mathematics, 1997, 40 :813-816
[42]   Stability of Hopfield-type neural networks (I) [J].
廖晓昕 .
Science China Mathematics, 1995, (04) :407-418
[43]   Daydreaming Hopfield Networks and their surprising effectiveness on correlated data [J].
Serricchio, Ludovica ;
Bocchi, Dario ;
Chilin, Claudio ;
Marino, Raffaele ;
Negri, Matteo ;
Cammarota, Chiara ;
Ricci-Tersenghi, Federico .
NEURAL NETWORKS, 2025, 186
[44]   Retrieval Capabilities of Hierarchical Networks: From Dyson to Hopfield [J].
Agliari, Elena ;
Barra, Adriano ;
Galluzzi, Andrea ;
Guerra, Francesco ;
Tantari, Daniele ;
Tavani, Flavia .
PHYSICAL REVIEW LETTERS, 2015, 114 (02)
[45]   STABILITY ANALYSIS OF HOPFIELD NEURAL NETWORKS WITH TIME DELAY [J].
王林山 ;
徐道义 .
AppliedMathematicsandMechanics(EnglishEdition), 2002, (01) :65-70
[46]   Stability analysis of Hopfield neural networks with time delay [J].
Wang Lin-shan ;
Xu Dao-yi .
Applied Mathematics and Mechanics, 2002, 23 (1) :65-70
[47]   Conformal Prediction for Time Series with Modern Hopfield Networks [J].
Auer, Andreas ;
Gauch, Martin ;
Klotz, Daniel ;
Hochreiter, Sepp .
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
[48]   Circuit design and exponential stabilization of memristive neural networks [J].
Wen, Shiping ;
Huang, Tingwen ;
Zeng, Zhigang ;
Chen, Yiran ;
Li, Peng .
NEURAL NETWORKS, 2015, 63 :48-56
[49]   Markovian jump delayed Hopfield networks with multiplicative noise [J].
Stoica, Adrian-Mihail ;
Yaesh, Isaac .
AUTOMATICA, 2008, 44 (08) :2157-2162
[50]   Modeling distributed concept representation in Hopfield neural networks [J].
Carvalho, LAV .
MATHEMATICAL AND COMPUTER MODELLING, 1999, 30 (1-2) :225-242