MASSIVELY PARALLEL ARCHITECTURES FOR LARGE-SCALE NEURAL NETWORK SIMULATIONS

被引:13
作者
FUJIMOTO, Y
FUKUDA, N
AKABANE, T
机构
[1] SHARP CO LTD,INTEGRATED CIRCUITS GRP,CTR IC DEV,RES STAFF,TENRI,NARA 632,JAPAN
[2] SHARP CO LTD,CORP RES & DEV GRP,CTR INFORMAT SYST RES & DEV,TENRI,NARA 632,JAPAN
来源
IEEE TRANSACTIONS ON NEURAL NETWORKS | 1992年 / 3卷 / 06期
关键词
D O I
10.1109/72.165590
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
A toroidal lattice architecture (TLA) and a planar lattice architecture (PIA) as massively parallel architectures of neurocomputers for large scale neural network simulations are proposed. The performances of these architectures are almost proportional to the number of node processors and they adopt the most efficient two-dimensional processor connections to be implemented by the wafer scale integration (WSI) technology to date. They also give a solution to the connectivity problem, the performance degradation caused by the data transmission bottleneck, and the load balancing problem for efficient parallel processing in large scale neural network simulations. Furthermore, these architectures will have an even greater expandability of parallelism and exhibit great flexibility for the various configurations of neural networks and varieties of neuron models. First we define the general neuron model that is the basis of these massively parallel architectures. Then, we take a multilayer perceptron (MLP) as a typical example of neural networks and describe the simulation of the MLP using error back propagation learning algorithms on virtual processors (VP's) with the TLA and the PLA. Then, the mapping from the VP's to physical node processors with the same TLA and PLA is presented. This mapping is done by row and column partitions. At the same time, the row and column permutations are carried out for node processor load balancing. The mapping algorithm for the load balancing is given. An equation to estimate the performance of these architectures is also presented. Finally, we describe implementation of the TLA with transputers including a parallel processor configuration, load balance algorithm, and evaluation of its performance. We have implemented a Hopfield neural network and a MLP, and applied them to the traveling salesman problem (TSP) and the identity mapping (IM), respectively. The TLA neurocomputer has achieved 2 MCPS in a feedforward network and 600 KCUPS in a back propagation network using 16 transputers. Actual proof that its performance increases almost in proportion to the number of node processors is given.
引用
收藏
页码:876 / 888
页数:13
相关论文
共 23 条
  • [1] BEYNON T, 1987, PARALLEL PROGRAMMING, P108
  • [2] BLELLOCH G, 1987, 10TH P INT JOINT C A, P323
  • [3] FUJIMOTO Y, 1989, JUN P IJCNN WASH, V2, P614
  • [4] FUJIMOTO Y, 1990, JUN P IJCNN SAN DIEG, V2, P581
  • [5] FUKUDA N, 1990, JAN P IJCNN, V2, P43
  • [6] GRAJSKI KA, 1990, JUL P INNC PAR
  • [7] HAMMERSTROM D, 1990, P INT JOINT C NEUR N, V2, P537
  • [8] HIRAIWA A, 1990, P IJCNN WASHINGTON D, V2, P137
  • [9] A QUANTITATIVE DESCRIPTION OF MEMBRANE CURRENT AND ITS APPLICATION TO CONDUCTION AND EXCITATION IN NERVE
    HODGKIN, AL
    HUXLEY, AF
    [J]. JOURNAL OF PHYSIOLOGY-LONDON, 1952, 117 (04): : 500 - 544
  • [10] IWATA A, 1989, JUN P IJCNN WASH, V2, P171