On the implicit acquisition of a context-free grammar by a simple recurrent neural network

The performance of a simple recurrent neural network on the implicit acquisition of a context-free grammar is re-examined and found to be significantly higher than previously reported by Elman. This result is obtained although the previous work employed it multilayer extension of the basic form of simple recurrent network and restricted the complexity of training and test corpora. The high performance is traced to a well-organized internal representation of the grammatical elements, as probed by a principal-component analysis of the hidden-layer activities. From the next-symbol-prediction performance on sentences not present in the training corpus, it capacity of generalization is demonstrated. (C) 2007 Elsevier B.V. All rights reserved.