Different eigenvalue distributions encode the same temporal tasks in recurrent neural networks

被引:0
|
作者
Cecilia Jarne
机构
[1] Departmento de Ciencia y Tecnología de la Universidad Nacional de Quilmes - CONICET,
来源
Cognitive Neurodynamics | 2023年 / 17卷
关键词
Dynamics; Recurrent neural networks; Eigenvalue spectrum;
D O I
暂无
中图分类号
学科分类号
摘要
Different brain areas, such as the cortex and, more specifically, the prefrontal cortex, show great recurrence in their connections, even in early sensory areas. Several approaches and methods based on trained networks have been proposed to model and describe these regions. It is essential to understand the dynamics behind the models because they are used to build different hypotheses about the functioning of brain areas and to explain experimental results. The main contribution here is the description of the dynamics through the classification and interpretation carried out with a set of numerical simulations. This study sheds light on the multiplicity of solutions obtained for the same tasks and shows the link between the spectra of linearized trained networks and the dynamics of the counterparts. The patterns in the distribution of the eigenvalues of the recurrent weight matrix were studied and properly related to the dynamics in each task.
引用
收藏
页码:257 / 275
页数:18
相关论文
共 39 条
  • [1] Different eigenvalue distributions encode the same temporal tasks in recurrent neural networks
    Jarne, Cecilia
    COGNITIVE NEURODYNAMICS, 2023, 17 (01) : 257 - 275
  • [2] Optimizing Recurrent Spiking Neural Networks with Small Time Constants for Temporal Tasks
    Zeng, Yuan
    Jeffs, Edward
    Stewart, Terrence C.
    Berdichevsky, Yevgeny
    Guo, Xiaochen
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON NEUROMORPHIC SYSTEMS 2022, ICONS 2022, 2022,
  • [3] Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks
    Jarne, Cecilia
    Laje, Rodrigo
    JOURNAL OF COMPUTATIONAL NEUROSCIENCE, 2023, 51 (04) : 407 - 431
  • [4] Temporal-Kernel Recurrent Neural Networks
    Sutskever, Ilya
    Hinton, Geoffrey
    NEURAL NETWORKS, 2010, 23 (02) : 239 - 243
  • [5] Training spatially homogeneous fully recurrent neural networks in eigenvalue space
    Perfetti, R
    Massarelli, E
    NEURAL NETWORKS, 1997, 10 (01) : 125 - 137
  • [6] Partially recurrent neural networks for production of temporal sequence
    Araujo, AFR
    D'Arbo, H
    IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE, 1998, : 474 - 479
  • [7] Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks
    Cecilia Jarne
    Rodrigo Laje
    Journal of Computational Neuroscience, 2023, 51 : 407 - 431
  • [8] Weighted automata extraction and explanation of recurrent neural networks for natural language tasks
    Wei, Zeming
    Zhang, Xiyue
    Zhang, Yihao
    Sun, Meng
    JOURNAL OF LOGICAL AND ALGEBRAIC METHODS IN PROGRAMMING, 2024, 136
  • [9] Learning deep hierarchical and temporal recurrent neural networks with residual learning
    Tehseen Zia
    Assad Abbas
    Usman Habib
    Muhammad Sajid Khan
    International Journal of Machine Learning and Cybernetics, 2020, 11 : 873 - 882
  • [10] Continual learning with attentive recurrent neural networks for temporal data classification
    Yin, Shao-Yu
    Huang, Yu
    Chang, Tien-Yu
    Chang, Shih-Fang
    Tseng, Vincent S.
    NEURAL NETWORKS, 2023, 158 : 171 - 187