Training of Elman networks and dynamic system modelling

A dynamic backpropagation (DBP) algorithm is presented to train the Elman network to model dynamic systems. The relationship between the Elman network trained by the DBP algorithm and the modified Elman network previously proposed by the authors is clarified. The paper shows that the modified Elman network is an approximate realization of the Elman network trained by the DBP algorithm. It is the self-feedback links of the context units of the modified Elman network which provide a dynamic trace of the gradients in the parameter space and enable the network to model dynamic systems of orders higher than one. The paper first gives the results of modelling a second-order linear plant and a third-order linear plant. Neither plant could be modelled using Elman networks trained by the standard backpropagation algorithm, but both were successfully modelled by DBP-trained Elman networks as they had been in previous studies by modified Elman networks. Finally, the paper reports on the application of the DBP-trained Elman net to model a benchmark nonlinear process.