COMPLEX TEMPORAL SEQUENCE LEARNING BASED ON SHORT-TERM-MEMORY

We design neural networks to learn, recognize, and reproduce complex temporal sequence, with short-term memory (STM) modeled by units comprising recurrent excitatory connections between two neurons (a dual neuron model). The output of a neuron has graded values instead of binary ones. By applying the Hebbian learning rule at each synapse and a normalization rule among all synaptic weights of a neuron, we show that a certain quantity, called the input potential, increases monotonically with sequence presentation, and that the neuron can only be fired when its input signals are arranged in a specific sequence. These sequence–detecting neurons form the basis for our model of complex sequence recognition, which can tolerate distortions of the learned sequences. A recurrent network of two layers is provided for reproducing complex sequences. © 1990 IEEE