Path integral approach to universal dynamics of reservoir computers

被引:1
作者
Haruna, Junichi [1 ]
Toshio, Riki [1 ]
Nakano, Naoto [2 ]
机构
[1] Kyoto Univ, Dept Phys, Kyoto 6068502, Japan
[2] Meiji Univ, Grad Sch Adv Math Sci, Tokyo 1648525, Japan
关键词
ECHO STATE NETWORKS; NEURAL-NETWORKS; FIELD-THEORY; CHAOS; EDGE; COMPUTATION; MEMORY; RENORMALIZATION; PREDICTION; PATTERNS;
D O I
10.1103/PhysRevE.107.034306
中图分类号
O35 [流体力学]; O53 [等离子体物理学];
学科分类号
070204 ; 080103 ; 080704 ;
摘要
In this work, we give a characterization of the reservoir computer (RC) by the network structure, especially the probability distribution of random coupling constants. First, based on the path integral method, we clarify the universal behavior of the random network dynamics in the thermodynamic limit, which depends only on the asymptotic behavior of the second cumulant generating functions of the network coupling constants. This result enables us to classify the random networks into several universality classes, according to the distribution function of coupling constants chosen for the networks. Interestingly, it is revealed that such a classification has a close relationship with the distribution of eigenvalues of the random coupling matrix. We also comment on the relation between our theory and some practical choices of random connectivity in the RC. Subsequently, we investigate the relationship between the RC's computational power and the network parameters for several universality classes. We perform several numerical simulations to evaluate the phase diagrams of the steady reservoir states, common-signal-induced synchronization, and the computational power in the chaotic time series inference tasks. As a result, we clarify the close relationship between these quantities, especially a remarkable computational performance near the phase transitions, which is realized even near a nonchaotic transition boundary. These results may provide us with a new perspective on the designing principle for the RC.
引用
收藏
页数:25
相关论文
共 90 条
  • [1] Properties of networks with partially structured and partially random connectivity
    Ahmadian, Yashar
    Fumarola, Francesco
    Miller, Kenneth D.
    [J]. PHYSICAL REVIEW E, 2015, 91 (01):
  • [2] Transition to Chaos in Random Networks with Cell-Type-Specific Connectivity
    Aljadeff, Johnatan
    Stern, Merav
    Sharpee, Tatyana
    [J]. PHYSICAL REVIEW LETTERS, 2015, 114 (08)
  • [3] Altland A., 2010, Condensed Matter Field Theory
  • [4] CHARACTERISTICS OF RANDOM NETS OF ANALOG NEURON-LIKE ELEMENTS
    AMARI, S
    [J]. IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS, 1972, SMC2 (05): : 643 - &
  • [5] [Anonymous], To get our numerical codes, please visit the GitHub page
  • [6] [Anonymous], 1979, Lecture Notes in Mathematics
  • [7] Beggs JM, 2003, J NEUROSCI, V23, P11167
  • [8] The criticality hypothesis: how local cortical networks might optimize information processing
    Beggs, John M.
    [J]. PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES, 2008, 366 (1864): : 329 - 343
  • [9] Reconciling modern machine-learning practice and the classical bias-variance trade-off
    Belkin, Mikhail
    Hsu, Daniel
    Ma, Siyuan
    Mandal, Soumik
    [J]. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2019, 116 (32) : 15849 - 15854
  • [10] Real-time computation at the edge of chaos in recurrent neural networks
    Bertschinger, N
    Natschläger, T
    [J]. NEURAL COMPUTATION, 2004, 16 (07) : 1413 - 1436